Derives a joint loss characterization in learning-rate and batch-size schedules, with a closed-form optimal batch-size schedule for any prescribed LR schedule; dynamic schedules beat static baselines in LLM training. A practical pretraining knob complementing the optimizer-comparison line (NVIDIA's SOAP/Muon study). Li and Bu (Meta).

Paper

scalingtrainingresearch