The Muon variant cluster (2026-09 → 2026-10)
paperYour notes
A cluster entry, not a paper. Since Muon was shown to scale and Kimi K2 trained on it, the optimizer has become a research literature of its own: enumerating cs.LG and cs.CL for September and October 2026 — 7,626 papers — turned up 17 untracked Muon papers in two months, against 25 entries that already mention it. A fourth variant of a known optimizer does not change how a frontier lab builds anything, so these are recorded here together rather than as seventeen thin files. What follows is the shape of the family; the weight of evidence is the signal, not any single entry in it.
Theory and guarantees. Finite-step Newton–Schulz orthogonalization now has training-loss guarantees (2610.03306), and momentum preconditioning has been extended beyond convex and gradient-Lipschitz potentials (2610.02158).
Mechanism — why it works at all. Spectral orthogonalization is credited with better factual learning (2610.02798), while small-batch noise is reported to restore the simplicity bias that parallel training erodes (2610.16868).
Structure-aware variants. Monarch-Muon targets memory and speed in pretraining (2610.17429); ExpertMuon-Compass sets alignment-guided step sizes for MoE training (2610.04140); Spectrally Targeted Muon (2610.10965), Muon-C for convolutional kernels (2609.09676) and AF-Muon for tied-embedding models without AdamW (2610.01395) round it out. Stability work: Musec momentum spectral clipping (2609.11655) and a Hilbert-geometry treatment of softmax at Muon's limits (2610.17588, MBZUAI).
The two most useful papers are the ones that push back. Muon Is Theoretically Wrong For Convolutions, But Empirically Effective (2610.07103, IRIT/UT3) shows that reshaping 4-D convolutional kernels into matrices breaks Muon's theoretical basis — then finds that Conv-NS, the theoretically sound replacement, performs equivalently on CIFAR-10 and ImageNet, hypothesising that exact convolutional orthogonalization overconstrains the update. From Geometry to Generalization (2610.11309) reports plain row normalization beating full-batch Adam and exact-SVD Muon. Read its scope carefully: that holds under an isotropic Gaussian-cloud model with synthetic experiments and last-layer language-model tests, and the advantage reverses when both covariances are diagonal and similar. It is a theoretical boundary condition, not a frontier-scale refutation, and should not be cited as one.
Settings beyond pretraining, listed for completeness and largely out of scope on their own: differentially private Muon (2610.02738), federated fine-tuning (2609.06073), LoRA tangent-space descent (2609.02734), and continual learning (2609.24678).
Attach later Muon variants to this entry rather than filing them. None of the seventeen reports a frontier-scale pretraining run; the papers that would change practice are the mechanism and limits results, which is why they are the two written out above.