A 4B coding agent from the Froggy team at Microsoft Research Montréal, post-trained from Qwen3.5-4B exclusively with RL on roughly 1,500 software-engineering environments — and notably without any distillation from a larger model, which is the usual way small agents are made. It resolves 61.5% of SWE-bench Verified, 37.6% of SWE-bench Pro, 31.1% of Terminal-Bench v2 and 47.3% of PatchEval-Verified, averaged over three runs, beating models an order of magnitude larger. Weights are on HuggingFace under MIT, and the community has already produced GGUF quantizations — fitting for a model whose pitch is running on minimal hardware.

The central idea is TaskPilot, an online task-synthesis loop: each iteration generates about 300 fresh synthetic tasks calibrated to the learnability frontier of the current checkpoint, then trains on them, then regenerates against the improved policy. The claim the report defends is not just that synthetic tasks suffice, but that recalibrating them to the agent as it improves is what makes them work — the comparison line uses real tasks from SWE-rebench and Scale-SWE, filtered by learnability for the same 4B policy. Training is continuous group-relative RL (DPPO) in 200-update climbs, policy weights carrying across iterations while optimizer and RNG state reset, on a single node of 8 NVIDIA B200s — two training with context parallelism, six serving rollouts, 32 task groups of 8 trajectories per update.

A finding worth more than the headline: the harness was a bottleneck before the model was. Their initial R2E-Gym setup, with its elaborate workflow, multi-purpose editor and dedicated finish action, was something Qwen3.5-4B could not reliably follow, so they built a simpler one called Leaf. Interface choices that frontier models absorb invisibly turn into failure modes at 4B. The efficiency numbers follow the same theme — FrogNano averages 87.1 turns against the base model's 100.5, hits the turn cap on 9.8% of trajectories versus 31.6%, and never overflows the context window where the base does. The report is also candid about artifacts and risk: an early checkpoint opened by searching for TypeScript files in 82.27% of cases despite Python-only training data (surviving into only 4% of evaluation trajectories), and the limitations section states plainly that rewards and scores rest on tests that cover only part of a program's behaviour, so a passing patch can still be incorrect or insecure, with agents trained this way liable to misread requirements, invent APIs, make overly broad edits and introduce regressions. Scope is English-language, Python-heavy repositories; generalization beyond that, and to other tool interfaces and budgets, is explicitly not established.

Model Details

Training hardware 8x NVIDIA B200
License MIT
Base model qwen3.5

Benchmark Scores

Benchmark Score Mode
SWE-bench Verified 61.5% —
SWE-bench Pro 37.6% —
Terminal-Bench v2 31.1% —
PatchEval-Verified 47.3% —

Paper

agentscodereinforcement-learningpost-trainingopen-weightresearch

Related