Training on the Test Task Confounds Evaluation

Moritz Hardt's benchmark-science anchor (ICLR 2025): apparent benchmark gains and "emergent" capabilities often reflect tuning on the test task rather than capability differences — comparing models after equalizing test-task exposure erases much of the claimed progress. Companion work ("Limits to Scalable Evaluation at the Frontier", ICLR 2025 Oral) shows LLM-as-judge debiasing cannot beat doubling human labels once the judge is no better than the evaluatee.