If you were to build a hardware rig to serve this at Q8, what would you rack up? It seems like "a ton of 96gb blackwells" is the answer for the uber-rich, but I'm wondering what other configurations might get the job done.
That's almost exactly what I'm estimating based on Deepseek R1 pulling about 1 or 2 TPS. Offloading a few layers to some A6000s didnt really help much either.