Nanbeige4.2-3B
modelYour notes
A new base trained from scratch on 28T tokens as a looped transformer. The report is titled "Unlocking Agentic Capabilities in a Compact Model". Its 22 physical layers run twice per forward pass, giving 44 effective layers, with hidden size 3072, 48 query heads and 8 KV heads. The model has 4.17B parameters in total, or 3B without embeddings. The report says it chose looping from scratch over upcycling an existing model, and a full two-pass loop over sharing the KV cache across loops. The instruct model has a 256K context.
Self-reported results:
- Reasoning: GPQA-Diamond 87.4 and HMMT Feb 2026 82.8.
- Agentic coding: SWE-bench Verified 63.6, SWE-bench Pro 46.9 and Terminal-Bench 2.0 44.1.
- Other: GDPval rubrics 74.3 and AA-LCR 58.7.
Artificial Analysis's August 2026 study of models that fit on a phone scored it 63 on its mobile-device benchmark set at 16K context, tied for first. That set is separate from the Intelligence Index.
The release added:
- FP8 and GPTQ-Int8 checkpoints.
- Nanbeige forks of vLLM, SGLang, llama.cpp and Ollama to serve the looped architecture.
- DSpark (August 31): an 848M speculative-decoding draft model that reads hidden states from five target layers and drafts seven tokens at a time.
The card says its modeling code already contains LoopSplit, mHC with depth attention and concatenated n-gram embeddings. These are features meant for Nanbeige4.5, which was in training for release later in 2026.
Model Details
Benchmark Scores
| Benchmark | Score | Mode |
|---|---|---|
| GPQA Diamond | 87.4 | — |
| SWE-bench Verified | 63.6 | — |
| SWE-bench Pro | 46.9 | — |
| Terminal-Bench 2.0 | 44.1 | — |
| AA mobile-device benchmark set (16K) | 63 | — |
Variants
| Name | Parameters | Notes |
|---|---|---|
| Nanbeige4.2-3B-Base | — | Base model, 128K context |
| Nanbeige4.2-3B | — | Instruct/reasoning model, 256K context; FP8 and GPTQ-Int8 checkpoints |
| Nanbeige4.2-3B-DSpark | — | 848M speculative-decoding draft model, 7-token blocks (Aug 31) |