Muse Glimmer
modelYour notes
Meta Superintelligence Labs' first open-weight Muse model — a 30B dense, natively multimodal model built for always-on, on-device agentic workflows, released Apache 2.0 on August 10, 2026. Its pretraining is logit distillation from Meta's proprietary Muse Spark into freshly initialised weights rather than a fine-tune of an existing checkpoint, then aligned with SFT, on-policy distillation and RL. Architecturally it is a "Dense Causal Transformer with Perception Encoder" (52 layers, ~29.6B language-model params plus a ~1.8B ViT-G/14 vision encoder), 128K context, text and image in — a Meta-native stack (its own muse_glimmer architecture and 202K vocabulary), not a fork of another lab's weights.
On Meta's own card it leads Gemma 4 31B across the board and trades with Qwen3.6-27B among ~30B open models: SWE-Bench Verified 76.0, SWE-Bench Pro 51.2 (#1), AIME 2026 94.7 (#1), GPQA Diamond 83.5, MMMU Pro 74, and strong agentic tool use (MCP Atlas 75.5, DeepSearch QA 74.6, both #1); Terminal-Bench 2.1 51.7 trails Qwen's 60.7. Ships as BF16 plus 4-bit K-quants and a DFlash speculative-decoding drafter for low-latency local inference — the first open-weight entry in the otherwise-proprietary Muse line (Spark, Muse Code). AA had not yet scored it at release; it now scores 6 on AA v5.0.1.
The technical report (arXiv 2610.24628, posted 2026-10-17, ten weeks after the weights) explains the two design choices that the card only stated. Dense over MoE was a measured decision, not a default. Meta trained scaling ladders at a fixed 32GB inference-memory ceiling: a dense ladder at 550 tokens-per-parameter against 1:8 and 1:16 MoE ladders at 2,800 and 6,000 TPP. Under that constraint dense was 2–2.5× more compute-efficient. That runs against the 2026 default, and it is scoped to memory-bound on-device targets, where the MoE's total parameters are the cost. Pretraining was co-distillation. The top-256 logits were logged from Muse Spark during Spark's own pretraining, and the student was trained on KL alone for the first 95% of the run, then on cross-entropy for the last 5% to correct top-k truncation bias. Four checkpoints spaced 600 steps apart were souped to initialise mid-training. Architecture: 52 layers, hidden 6656, GQA 32:2, a 3-local:1-global sliding-window pattern (window 2048) with gated attention, and the Llama 4 BPE tokenizer (202,048 tokens). The report does not give a total token count or hardware. Its quantisation study puts the BF16 average at 75.7 across 15 benchmarks against 74.8 for Q4_K_M and 74.4 for NVFP4/MXFP8.
Model Details
Benchmark Scores
| Benchmark | Score | Mode |
|---|---|---|
| SWE-Bench Verified | 76.0 | — |
| SWE-Bench Pro | 51.2 | — |
| Terminal-Bench 2.1 | 51.7 | — |
| GPQA Diamond | 83.5 | — |
| AIME 2026 | 94.7 | — |
| MCP Atlas | 75.5 | — |
| MMMU Pro | 74 | — |