Hy4 preview

Tencent Hy Team's next-generation open-weight flagship, shipped preview-first under eight weeks after Hy3 went GA, and the second generation built under Chief AI Scientist Yao Shunyu. 770B total / 49B active MoE (the safetensors checkpoint totals ~780B, which also counts a native 10B-parameter / 0.7B-active MTP layer for speculative decoding): 78 layers, hidden size 6144, a dense FFN in layer 1 and 77 MoE layers with 256 routed + 1 shared expert, top-8 routing, vocab 120,832. It is a ground-up architecture change from Hy3 (295B/21B, 192 experts, 80 layers, GQA): attention is Gated DeepSeek Sparse Attention over MLA (query/KV compression ranks 2048/512, 64 heads) with a 32-head indexer selecting the top-2048 tokens per query, reusing indices across layers via IndexCache (one "full" indexer per four layers, the rest shared, per config.json), and the residual path uses iHC (identity Hyper-Connections) with 4 residual streams (cf. DeepSeek's mHC). Context is 1M tokens (max_position_embeddings 1,048,576), up from Hy3's 256K. Tencent says it "scaled Hy4 preview on three fronts: model size, context length, and training data"; the training token count, hardware and duration are undisclosed. The open preview is text-only (HF pipeline text-generation, OpenRouter text→text) — no vision or audio variant has shipped. Apache-2.0, BF16 and FP8 checkpoints, with day-0 vLLM and SGLang images.

Model-card appendix (highest reasoning setting; most agentic evals run by Tencent through the Claude Code harness), Hy4 preview vs Hy3: SWE-bench Multilingual 82.9 (75.8), SWE-bench Pro 65.7 (57.9), Terminal-Bench 2.1 85.4 (70.8), DeepSWE 64.3 (28.0), CyberGym 78.4 (51.8), Toolathlon-Verified 74.1 (56.2), MCP-Atlas 83.7 (75.0), GDPval-AA V2 Elo 1678 (1213), GPQA Diamond 92.3 (90.9), HLE no-tools 43.4 (34.4), MathArena Apex 2025 74.2 (38.7), BioMysteryBench 71.3 (54.9), CritPt 16.9 (4.9). That puts it at rough parity with GLM 5.3, Kimi K3 and DeepSeek V4 Pro on most rows, while Claude Opus 5 still leads clearly on SWE-bench Pro (79.2), SWE-Marathon (50.0 vs 31.9) and ProgramBench (39.5 vs 17.5). In a blind internal side-by-side, 163 Tencent experts rated it 2.99/4 on 203 engineering tasks vs GLM 5.3's 2.92 and Kimi K3's 2.94. Tencent also says the model helped optimize its own training methods, data strategy, evals and low-level operators and lifted its own inference throughput 31.8%, calling this an "early-stage recursive self-improvement loop". Known limitations: overlong reasoning and over-verification. API pricing is $0.834/$2.501 per M input/output tokens ($0.042 cache hits) via Tencent Cloud TokenHub and OpenRouter, free for two weeks in WorkBuddy and CodeBuddy; "the next batch of models in the Hy4 series is expected to roll out soon", following the Hy3 preview→GA pattern. Not yet scored on the Artificial Analysis Intelligence Index as of 2026-08-29.