Understanding Emergent Abilities from the Loss Perspective

Theoretical analysis of emergent abilities in language models through the lens of pre-training loss, providing a unified framework for understanding when and why capabilities appear at scale. Published at NeurIPS 2024.