MathCoder2 / MathCode-Pile

MathCode-Pile: a 19.2B-token mathematical continued-pretraining corpus built by pairing math web/textbook/synthetic data with model-translated mathematical code — LaTeX expressions extracted and re-expressed as executable code with reasoning steps — plus the MathCoder2 model family continued-pretrained on it. The corpus became one of the widely reused open math pretraining datasets (ICLR 2025).

From Hongsheng Li's MMLab-CUHK / MathLLM group. The models are continued-pretrains of open bases (Llama-3, DeepSeekMath, Mistral, Code-Llama), not from-scratch. Successors: MathCoder-VL (ACL 2025). The original 2023 MathCoder is filed under Shanghai AI Lab per its lead affiliation.