A unified multimodal diffusion language model that covers text QA, text-to-image generation, image understanding and editing, video understanding, and 3D understanding in one 16B-A1B MoE. Its language backbone is Ant Group's LLaDA-2.0 block-diffusion MoE, extended to vision through a fully discrete multi-codebook representation: Apple's AToken tokenizer describes each visual position with eight complementary codebooks, whose embeddings are concatenated into one backbone token on input and decoded by a codebook-depth autoregressive head with spatially parallel denoising on output. The inference code landed in August; the weights released on September 10 were the Ascend-trained checkpoint, and the promised NVIDIA-trained twin followed on September 24 in the same HuggingFace repo (nvidia/, seven bf16 shards, alongside ascend/). Shanghai AI Lab reports operator-level numerical alignment between the two backends.

Preliminary results at the September release: ChartQA 86.52, CharXiv-DQ 83.65, MathVision 33.22, DynaMath 56.37. Apache 2.0.

The technical report (arXiv 2610.20818, October 2026) calls it, "to our knowledge the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space", built on AToken's unified 4D representation so image, video and 3D inputs share one backbone. It describes the full-stack training, inference and evaluation system on both Ascend and NVIDIA, and a stability-aware sampling strategy (confidence-plateau-guided unmasking plus drift-triggered cache refresh) that it reports accelerates image generation 4.45×. The GitHub repo added multi-task SFT training code and the optional inference-acceleration path on October 16 and the Ascend NPU training stack on October 20.

Model Details

Architecture MOE
License Apache-2.0
Base model llada-2

Paper

multimodaldiffusionmoeopen-weight

Related