MOPD: Multi-Teacher On-Policy Distillation
paperYour notes
Xiaomi's answer to a problem that shows up once RL post-training works: you can push one capability at a time, but combining several into a single model is where recipes break down. The paper's framing is that the existing options each fail differently — Off-Policy Finetune and Mix-RL are either inefficient or lose performance, and parameter merging dilutes what each specialist learned.
Multi-teacher On-Policy Distillation (MOPD) splits the problem in two. First run per-domain specialised RL to get a set of domain teachers; then distill those teachers into the student on the student's own rollouts, which removes exposure bias and gives a dense per-token signal instead of an episode-level one. The structural payoff is as interesting as the scores: because each teacher is trained independently, domain work can proceed in parallel, removing the cross-domain coupling that makes multi-domain post-training a scheduling problem. On Qwen3-30B-A3B it beats Mix-RL, Cascade RL, Off-Policy Finetune and Param-Merge, and the paper reports the student inheriting nearly all of each teacher's capability.
The technique shipped before it was written up: it was used in the post-training of MiMo-V2-Flash in December 2025, which is the date this entry carries, and the dedicated paper (arXiv:2606.30406) followed in June 2026 — the authors cite that deployment as the evidence it holds at frontier scale. By Xiaomi with Peking University collaborators; senior author Fuli Luo.