DataComp-LM (DCLM-7B)

The open-data benchmark that set the standard for language-model pretraining data (NeurIPS 2024). DataComp-LM provides a 240T-token CommonCrawl testbed plus the DCLM-Baseline curation recipe; DCLM-7B, trained from scratch on the baseline data, reached 64% MMLU — the best open-data LM of its cohort — and drove wide adoption (mlfoundations/dclm, 1.4K★). Led by Ludwig Schmidt's mlfoundations group while at UW (Schmidt moved to Stanford/Anthropic in November 2024); the first-submission predates the tracking window by a month but the conference and adoption land in-window, and it is the core UW open-data artifact.