PKU-DS-LAB's (Tong Yang) branch-merge distillation applied to DeepSeek-R1-Distill-Qwen-32B: specialized math and code branches are trained separately, then merged, reaching roughly R1-671B-level AIME and LiveCodeBench scores at ~5% of the parameters (102 HF likes). The method paper is TinyR1-32B-Preview (joint with Qihoo360). The same lab followed with iFairy / Fairy2i, the first 2-bit complex-valued LLMs (weights in {±1, ±i}).

Model Details

License Apache 2.0
Base model deepseek-r1

Paper

reasoningopen-weightefficiency

Notes

The arXiv ID 2505.17320 sometimes cited for FairyR1 resolves to an unrelated TTS paper; the correct method paper is TinyR1 (arXiv:2503.04872).