FLAN (Instruction Tuning)

"Finetuned Language Models Are Zero-Shot Learners." Instruction-tuned a 137B parameter model on 60+ NLP tasks described via natural language instructions. FLAN surpassed zero-shot GPT-3 (175B) on 20 of 25 tasks despite being smaller.

Pioneered instruction tuning, demonstrating that fine-tuning on a diverse set of tasks with instructions dramatically improves zero-shot generalization. Now a standard technique across all frontier labs. ICLR 2022. By Wei, Bosma, Zhao et al.