The first open-weight model of Xiaohongshu dots studio's dots3 family, released August 14, 2026, a month after a sibling branch, dots-note-3.0, scored 42/42 at IMO 2026. The model card describes a natively multimodal MoE with 280B total and 16B activated parameters and a 512K-token context. It takes text, image, video and audio input and produces text. The language model has 1 dense and 45 MoE layers, 256 routed experts plus one shared with top-8 sigmoid routing, hidden size 5120 and a 152K vocabulary. Attention is a roughly 1:3 hybrid: 13 layers of DeepSeek-style sparse attention (DSA, top-2048 indexer over MLA) and 33 sliding-window layers, with headwise attention gates. A 1.13B shared MTP layer supports speculative decoding. The vision encoder is itself an MoE ViT (7B total, 1.2B active) and the audio encoder a dense 800M model. Weights ship in BF16 and FP8 under Apache 2.0, with vLLM support on main and Transformers and SGLang support in open PRs. The card calls it "the most lightweight member of the family". Larger jazz and aria tiers are planned, and launch coverage reports that a non-preview dots3-note is also due. Neither had appeared on the dots-studio HuggingFace org as of 2026-11-10.

The release targets long-horizon agent work rather than single-turn benchmarks. According to the tech blog as summarised in launch coverage, dots studio built thousands of long, novel environments that require no prior knowledge, and trained the model with RL to keep useful information once a task outgrows its context window. It also introduces TEMPO (Test-time-scaled Value Estimation with Macro-step Policy Optimization). TEMPO cuts a long trajectory into macro-steps, and at each boundary the policy switches to acting as a critic, using reasoning, tool calls and test-time scaling to estimate the return of the unfinished trajectory. That estimate becomes a training signal before the episode ends. The reported curves show GRPO and the base checkpoint plateauing as interaction turns grow while TEMPO keeps improving. The same self-critique loop is credited for the IMO result. Self-reported: about 0.35 on ARC-AGI-3 at under $500, below Claude Opus 4.8 (high) in absolute score. The lab released two agent evaluations with it, VibeSearchBench (200 tasks across 20 domains with requirements revealed over several turns) and VibeLifeBench (20 tasks of 20–30 stages with 1,247 atomic checks), and reports that no model, Claude Opus 5 and GPT-5.5 included, reaches its pass line on either. The card gives benchmark tables only as images. The full technical report is marked "coming soon". The card does not describe pretraining (token count, data, hardware). The architecture matches no published external checkpoint, so the entry records it as dots studio's own model, following the from-scratch dots.llm1 line.

Model Details

Architecture MOE
Parameters 280B
Active params 16B
Experts 256 (top-8)
Context window 524,288
License Apache 2.0

Variants

Name Parameters Notes
dots3-note-prev — BF16
dots3-note-prev-fp8 — FP8
moeopen-weightmultimodalagenticreasoninglong-contextrl

Related