The large open-weights member of the Nemotron 3.5 line and the one that carries the architecture argument: 120B total / 12B active in a Mamba-2 – Transformer hybrid MoE with 512 routed experts per MoE layer, over a 1M-token context, taking text, images and video. The hybrid-SSM direction that Nemotron 3.5 Lightning ran at 30B/3B is here at four times the scale, with native Multi-Token Prediction for speculative decoding and configurable reasoning modes. The foundation corpus is over 20 trillion tokens across English, 19 further languages and 43 programming languages, plus synthetic data from several stronger models.

NVIDIA reports across 50-plus benchmarks; the representative set is GPQA-Diamond 89.52, LiveCodeBench v6 88.22, WMT24++ 84.80, MathVision 78.16, MMLU-ProX 76.91, SWE-bench Verified 71.10 and SWE-bench Pro 51.78. Two of those deserve reading together: SWE-bench Verified at 71.10 is frontier-adjacent for an open model, while Video-MME v2 at 32.85 is low for a model that lists video as a first-class input — the vision-language claim is much stronger on documents and stills than on video, and the published table says so.

Released under OpenMDW-1.1, a genuine open licence rather than a revenue-gated one — a different posture from Muse Spark Open, which shipped the same week under a bespoke licence with a $20M notification clause.

Artificial Analysis scored it on release day at 26 on Intelligence Index v4.3.2, #1 of 67 open-weights models in its size class and, in AA's words, "the leading US open weight model" — twice the 13 of its predecessor Nemotron 3 Super on that version. On v5.0 (October 27) it reads 14. Two figures make the efficiency case as well as the score does: it spends 46k output tokens per index task, about 60% fewer than its predecessor, and serves at ~630 tok/s on 100k-token prompts with a 1.8s first token, at $0.20 / $0.57 per 1M.

The joint technical report for Super and Lightning (arXiv 2610.38095, October 28, 2026) describes both as built with large-scale distillation, long-context training and native speculative decoding. Super adds Indic continued pretraining, which NVIDIA says lifts Indian-language benchmarks by up to 9.82 points, and agentic software-engineering mid-training. Its post-training runs SFT, RLVR and RLHF plus specialized teachers and multi-teacher on-policy distillation, which is what turns the text-only base into a multimodal agent. The report gives Super 51.78% on SWE-bench Pro, above Nemotron 3 Ultra with about 4.6× fewer active parameters. MTP boosting after post-training lengthens speculative-decoding acceptance by up to 21.8%, NVFP4 keeps a 99.62% median accuracy recovery against BF16, and serving reaches up to 4,856 output tokens/s per GB200 GPU at 50+ tokens/s per user.

Model Details

Architecture MOE
Parameters 120B
Active params 12B
Context window 1,000,000
AA Intelligence 14 was 26 on v4.3.2
License OpenMDW-1.1

Benchmark Scores

Benchmark Score Mode
GPQA-Diamond 89.52 —
LiveCodeBench v6 88.22 —
WMT24++ 84.80 —
MathVision 78.16 —
MMLU-ProX 76.91 —
SWE-bench Verified 71.10 —
SWE-bench Pro 51.78 —
Video-MME v2 32.85 —

Variants

Name Parameters Notes
Super VL 120B-A12B BF16 — full-precision reference weights
Super VL 120B-A12B Base BF16 — base (pre-post-training) checkpoint
Super VL 120B-A12B NVFP4 — NVFP4 quantization for deployment

Paper

frontieropen-weightsmoemultimodalreasoningefficiency

Related