LR Schedules and Convex Optimization

Francis Bach's SIERRA team (Schaipp, Taylor, Simsekli, Bach; ICML 2025): the learning-rate schedules used in large-model training — cosine, WSD, cooldown — match a nonsmooth-convex optimization bound, giving a principled account of a widely-used empirical practice and a recipe for tuning it. Companion NeurIPS 2025 result explains why sign-descent/Adam beats gradient descent on Zipfian token data.