Phi-1 ("Textbooks Are All You Need")

"Textbooks Are All You Need." 1.3B parameter code-focused model trained on high-quality "textbook-like" synthetic data. Achieved 50.6% on HumanEval despite being orders of magnitude smaller than competitors.

Phi-1 demonstrated that data quality matters more than data quantity for small models, launching the Phi series and influencing the industry's approach to synthetic data curation. By Gunasekar, Zhang, Anber et al. MIT License.