Multi-discipline olympiad-level benchmark for "superintelligent AI" cognitive reasoning (NeurIPS 2024 Datasets & Benchmarks): 11,163 bilingual problems drawn from 62 international olympic competitions across math, physics, chemistry, biology, geography, astronomy, and CS, with process-level as well as answer-level evaluation. From Pengfei Liu's pre-SII GAIR lab; picked up in frontier-model eval suites.

Paper

Venue NeurIPS 2024

Evaluation Details

Questions 11,163
Domains 7
Scoring answer-level and process-level evaluation across 62 olympiad competitions
Domains: mathematics, physics, chemistry, biology, geography, astronomy, computer science
benchmarkevaluationreasoning