EMNLP 2025: a 29-language parallel extension of the MMLU-Pro reasoning benchmark, built with a semi-automatic translation pipeline plus expert verification, and used to evaluate 36 state-of-the-art LLMs — exposing large high- vs low-resource language gaps. Multi-institution effort with Yusuke Iwasawa (Matsuo-Iwasawa Lab) as senior author; ~6.6K HF downloads signal real adoption as a multilingual-reasoning standard.

Paper

Venue EMNLP 2025

Evaluation Details

Domains 1
Domains: multilingual reasoning
evaluationmultilingual