K2 Horizon

K2 Horizon is a fleet of six fully open models from MBZUAI's Institute of Foundation Models (IFM), released September 3, 2026: 375B-A23B, MoVA 36B-A4B, 32B, 7B, 3.7B, and 0.9B, all under Apache 2.0 with a shared architecture, a 250K-token vocabulary, and a native 524,288-token context from mid-training onward (the 0.9B uses a 64K vocabulary and 128K context). Each model is pretrained on roughly 20 trillion tokens (the 3.7B, 7B, 32B, and 36B-A4B on an identical 22T-token stream); nearly 17% of the corpus is problem-solving trajectories with explicit reasoning, and about 10T tokens are synthetic. IFM commits to releasing intermediate checkpoints, training data or construction recipes, mixture compositions, the xLLM training stack, post-training and RL code, fine-grained logs, and evaluation results, extending the LLM360 "360-open" line behind K2-V2 and K2 Think V2. At launch the final weights are public and the checkpoints, data, and code are marked as released from August 24 as TxT360-v2 and the Code-, Math-, and SFT-Reasoning and Pretrain-Behaviors sets; training hardware and cost are not disclosed.

The flagship K2-Horizon-375B-A23B is a sparse MoE with 375B total and about 23B active parameters: 61 layers, hidden size 6,144, 48 query and 8 KV heads, 192 routed experts with top-8 sigmoid routing plus one shared expert, and dense MLPs in the first three layers. Post-training chains mid-training, SFT, model merging, and RL with agent-specialized branches over more than 100 million synthesized tasks, yielding a "development tree" of reasoning and agentic variants rather than a single chat model. At high reasoning effort it scores Terminal-Bench 2.1 70.2, GPQA Diamond 87.3, Humanity's Last Exam 32.0, SWE-Bench Pro 42.6, BrowseComp 72.8, Toolathlon 65.3, MCPMark 67.7, and a GDPval-AA Elo of 1,441, ahead of Nemotron 3 Ultra (550B-A55B) on every listed agentic benchmark and trailing GLM 5.2 (753B-A40B) on most rows while leading it on Toolathlon. IFM audited the Terminal-Bench run for reward hacking with Artificial Analysis's procedure: 24 of 500 passing trials were flagged, which lowers the score from 70.2 to 66.9. On the Artificial Analysis Intelligence Index v4.3 it scores 31 (34 when v4.3 launched on 8 Sep 2026; 38 on v4.2; 47 on v4.1.1 at release), 20 points above K2 Think V2's 11.

Two architectural additions ship with the fleet. MoVA (Mixture-of-Value Attention) brings expert routing into multi-head attention the way MoE sparsifies feed-forward layers; the 36B-A4B model pairs 100 feed-forward experts (top-8) with 64 attention-value experts (top-4) to land just below the dense 32B at 4B active parameters. Uno is a conditional LoRA adapter trained by "diffusion distillation" to emit blocks of tokens in parallel while the frozen autoregressive weights keep the output distribution, a lossless decode speedup that IFM's release puts at roughly 3×; adapters ship for the 7B and 0.9B models. Serving recipes exist for vLLM and SGLang (BF16, TP=8 on one 8×H200 node for the 375B), with FP8 and GGUF checkpoints for most sizes.