Modular full-stack RL library for LLM agents from the NovaSky team (co-developed with Anyscale): trained SkyRL-SQL-7B, SkyRL-Agent-8B, and SA-SWE-32B (39.4% SWE-bench Verified at over 2× lower training cost); the v0.1 "tx" engine adds a Tinker-compatible local RL backend. One of the three reference open agent-RL stacks alongside veRL and rLLM.

v0.4.0 (2026-10-01) targets large-scale post-training. It adds training support for GLM 5.2/5.3 and GLM 5.3 Flash, Kimi K2.6/2.7 (experimental), Qwen3.8 and Nemotron 3.5-Lightning. For Kimi K2.6/2.7 the team says LoRA training of the 1T-parameter model fits on two B300 nodes, with INT4 serving for rollouts and fake quantization on the trainer to limit the train/inference mismatch. On stability, it adds top-p/top-k sampler replay (the keep-sampling-mask approach from DeepSeek-V3.2) and a reworked rollout routing replay (R3). The team reports that R3 throughput on Nemotron 3 Super rose from 111 to 483 tokens/s/GPU. Blockwise FP8 and MXFP8 RL are also new. The team says FP8 tracks BF16 convergence while cutting step time by up to 23%. Weight sync gains XOR delta sync for disaggregated RL over ethernet, a sharded NIXL/RDT backend (Kimi K2 BF16 weights transferred in 7.53 s on 48 8×H100 nodes), and in-memory LoRA sync. The native SFT trainer now handles 10B–1T+ token datasets. All figures are the team's own release-note numbers; none is independently measured. Apache 2.0; about 2.4K GitHub stars.

Paper

Library

License Apache 2.0
post-traininginfrastructureagents