NVIDIA's open-weight speech-augmented language model (SALM) in the Nemotron 3.5 line, released October 28, 2026 on HuggingFace, NGC, build.nvidia.com and NeMo Speech. It joins an audio front end to the Nemotron 3.5 Lightning 30B-A3B hybrid Mamba-2/Transformer MoE: a "Parallel Expert Encoder" made of a 600M-parameter Transformer ASR encoder and the ~99M-parameter Nemotron-3-Diarization (Sortformer) speaker encoder, plus a CTC timestamp head. The card gives 34B total and 4.3B active parameters including the audio encoder (about 3.5B active in the language backbone and about 800M in the joint audio encoder), a 65,536-token context, recordings up to one hour, and 40 languages. It is a post-train of Lightning, not a new pretraining run: one SFT stage followed by "a short reinforcement learning stage".

One model covers verbatim and written-form transcription, contextual transcription with term hints, speaker-attributed transcription, speech translation, timestamps, language ID and English audio question answering. Training used about 1.72M unique audio hours (6.53M with repeated exposure) and 5.6B text tokens, much of it from the companion Granary-v2 multilingual speech dataset. Self-reported results, captured October 12, 2026: mean WER 4.71 on the 9 English Open ASR Leaderboard sets (Qwen3-ASR-1.7B 4.94), 4.62 on the 7-language multilingual leaderboard (Qwen3-ASR-1.7B 6.49), 10.98 long-form (Parakeet-TDT-0.6B-v3 13.37), and mean cpWER 15.80 on full-session multi-speaker meetings (AMI, CALLHOME, Mixer 6, NOTSOFAR-1; DiCoW v3.3 + DiariZen 16.83).

Serving uses joint multi-token prediction (two trained depths, up to eight speculative tokens) or a separate DFlash2 block-diffusion drafter. NVIDIA reports up to 2,238 seconds of audio transcribed per second on one RTX PRO 6000 Blackwell (NVFP4 with DFlash2) in its NIM, and notes those serving optimizations are not yet in the open-source release. Weights ship in BF16, FP8 and NVFP4 under OpenMDW-1.1.

Model Details

Architecture MOE
Context window 65,536
License OpenMDW-1.1
Base model nemotron-3.5-lightning

Benchmark Scores

Benchmark Score Mode
Open ASR Leaderboard (English, mean WER) 4.71 —
Multilingual Open ASR Leaderboard (mean WER) 4.62 —
Open ASR long-form (mean WER) 10.98 —

Variants

Name Parameters Notes
NVIDIA-Nemotron-3.5-Transcribe-30B-A4B — 34B total / 4.3B active including audio encoder; post-trained on Nemotron 3.5 Lightning 30B-A3B; BF16, FP8, NVFP4
speechaudiomultimodalopen-weightmoe

Related