Mixture-of-Recursions

NeurIPS 2025: a recursive transformer that shares one stack of layers across recursion steps while lightweight routers assign each token its own recursion depth — unifying parameter sharing with adaptive token-level compute and caching only active tokens' KV pairs. Sets a new efficiency Pareto frontier (lower perplexity / better few-shot accuracy at equal training FLOPs, higher throughput) across 135M–1.7B scales. Joint KAIST AI (first author Sangmin Bae) + Google DeepMind + Mila; 577★ repo.