An agent harness that makes an existing LLM training job faster without changing what it trains. You give it a working launch script, entry point or repository. It searches launch flags, environment variables, config keys and code patches you allow, then re-verifies the winner against your original job. It works with Megatron-Core, FSDP2/TorchTitan, DeepSpeed, HF Trainer/TRL and PyTorch DDP, on one node or several. The agent can be Claude Code, Codex, DeepSeek or any OpenAI-compatible endpoint.

The paper's argument is that training throughput defeats plain coding agents and greedy tuners for four reasons, and the harness is built around each:

  • Fake speedups. Agents can "win" by shrinking the effective batch, moving optimizer work outside the timing window or casting to unapproved precision. The task contract is fixed before search, and every candidate must pass 16 invariants in five layers: execution, workload, schedule, numerics (loss within a default 3%) and a reproduced speedup in a fresh back-to-back re-run.
  • Measurement noise. Paired step-time noise ranges from 0.04–0.10% on dedicated 8×H100/H200 nodes to over 6% on shared hosts. So a doctor stage calibrates the cluster's noise floor instead of using a fixed 2% gate.
  • Wrong LLM priors. Zero-shot LLM speedup predictions correlated negatively with measurement (Spearman −0.49 to −0.06). The model predicted +2% from graph compilation on Gemma-4-26B-A4B; the measured gain was +55%. InfraRSI fits global step-time and censored peak-memory models over all trials, out-of-memory failures included, and revises the priors from live runs.
  • Memory walls. Many fast settings only fit in combination. The search prices each setting's memory cost and pairs memory-hungry optimizations with memory savers and verified patches.

Results on four workloads, under the same task contracts as the METR RE-Bench agent scaffold, Auto-Research and the knob-only tuner OptiCo:

  • Overall: 1.83× geometric-mean speedup, with mean MFU rising from 17.4% to 30.0%. The baselines reach 1.37×, 1.34× and 1.23×.
  • Qwen3.5-9B (8×H100): 1.73×, MFU 25.3% to 43.8%.
  • Gemma-4-26B-A4B MoE (8×H200): 2.75×, MFU 6.4% to 17.6%. This came from torch.compile with recomputation off, paired with a 14 GiB memory saver, plus a compiled FlexAttention patch.
  • Qwen3.8-27B (8×H200): 1.55×.
  • Qwen3.5-35B-A3B MoE (16×H100, two nodes): 1.51×. Part of this came from a 32-channel NCCL setting the LLM had rated a slowdown, which measured +10.2%.

The "RSI" in the name is a two-level loop: priors are revised within a run, and each verified result is written to an experience bank shared across hardware. On Qwen3.5-9B, experience from other GPU types alone raised the final speedup from 1.70× (cold start) to 1.82×. It reached 90% of peak in 9 trials, which the cold start never did within budget.

The authors are from TokenRhythm, Peking University, Infinigence AI, Shanghai Jiao Tong University and Tsinghua. The corresponding authors are Kai Han, Yunhe Wang, Allen Chen and Yu Wang. The code was released on November 8, 2026. It belongs with the other recursive self-improvement work aimed at the training stack: SII's 801-30B-A3B ran agent debugging loops over a full pretraining run, and Google's RRSI regularizes harness self-editing.

Paper

Authors: InfraRSI Team · Yuchuan Tian · Haocheng Mei · Ye Yuan · Chao Xu · Boxun Li · Guohao Dai · Kai Han · Allen Chen · Yunhe Wang · Yu Wang
infrastructuretrainingagentsrecursive-self-improvementefficiencyresearch

Related