Tencent Hy4 Preview: Open 770B MoE Built for Real Work

Late August brought another big open-weight release from China, and this one is aimed squarely at people who ship software and run analyses for a living. On August 28, 2026, Tencent’s Hunyuan (Hy) team published Hy4 preview: a mixture-of-experts language model with 770 billion total parameters, about 49 billion active per token, a 1 million token context window, and weights under Apache 2.0.

Official Hy4 preview branding from Tencent’s Hugging Face model card (tencent/Hy4-preview).

That combination matters more than the headline parameter count. You get frontier-scale capacity with MoE routing that keeps per-token compute closer to a ~50B dense model, a license that allows commercial and on-prem use, and serving recipes for vLLM and SGLang that already include speculative decoding. Reuters also noted that Tencent plans to wire the model into products such as CodeBuddy and WorkBuddy, so the release is not only a research drop.

What Hy4 preview actually is

Hy4 preview is the early ship of Tencent’s next Hunyuan generation. The backbone is 78 layers deep. The first layer uses a dense feed-forward network; the remaining 77 are MoE layers. Each MoE layer has 256 routed experts plus 1 shared expert, and every token activates the top 8 routed experts along with the shared expert. Hidden size is 6144. Vocabulary size is 120,832.

Architecture choices borrow from recent open research. Attention uses Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache so sparse indices can be reused across layers, which is how a 1M context stays practical. Residuals use identity Hyper-Connections (iHC) with four residual streams to widen information flow between layers. Separately, a native Multi-Token Prediction (MTP) layer (about 10B total / 0.7B active) is built in for speculative decoding, and the official vLLM / SGLang images are set up to use it.

Weights are available as BF16 (tencent/Hy4-preview) and FP8 (tencent/Hy4-preview-FP8) on Hugging Face, ModelScope, GitCode, and CNB. Hosted API access is listed through Tencent Cloud TokenHub and OpenRouter. At launch, Tencent quoted API pricing around $0.834 / 1M input tokens, $2.501 / 1M output tokens, and $0.042 / 1M for cache hits. Self-hosting the Apache 2.0 weights is free aside from your own GPUs.

Why it matters right now

Open MoE flagships are no longer rare. What stands out about Hy4 preview is the product framing. Tencent says it co-created training data with internal experts in software engineering, gaming, finance, and security, then co-designed the model against CodeBuddy and WorkBuddy. In an internal blind evaluation with 163 experts and 203 engineering tasks, Hy4 preview scored an average of 2.99 / 4.00, slightly ahead of GLM 5.3 (2.92) and Kimi K3 (2.94).

Published generation-over-generation jumps versus Hy3 are large on agentic coding and tool use. Examples from Tencent’s appendix (and independent summaries of that appendix) include DeepSWE rising from 28.0 to 64.3, Terminal-Bench 2.1 from 70.8 to 85.4, SWE-bench Pro from 57.9 to 65.7, and MathArena Apex 2025 from 38.7 to 74.2. Knowledge-work Elo on GDPval-AA V2 moved from 1213 to 1678. Pure reasoning moved less (GPQA Diamond 90.9 → 92.3), which is a useful reminder that “bigger MoE” does not evenly lift every axis.

Benchmark overview image published with the Hy4 preview model card on Hugging Face.

For teams that care about licensing and data residency, Apache 2.0 plus FP8 serving recipes is the practical story. You can evaluate the model through an API first, then move workloads on-prem without renegotiating terms. That is still rarer than “open weights with a restrictive non-commercial clause,” and it is why Hy4 preview will show up in enterprise bake-offs even if closed models still win some pure quality comparisons.

How it works at a useful level

Mixture-of-experts routing is the core cost trick. Instead of activating all 770B parameters for every token, the router picks a small expert set (top-8 routed + shared). Capacity lives in the expert pool; compute stays closer to the activated subset. Sparse attention then attacks the other expensive part of long-context inference: comparing every token to every other token. Gated DSA plus IndexCache keeps long prompts from exploding memory the way dense full attention would.

Speculative decoding via the native MTP layer is the third lever. A small draft head proposes several tokens; the main model verifies them in parallel. When acceptance rates are decent, wall-clock latency drops without changing the final distribution much. That is why the official Docker commands wire speculative config in by default rather than treating it as an optional experiment.

Serving guidance is concrete. Tencent points people at 8-way tensor parallel FP8 serving with images such as vllm/vllm-openai:hy4-preview and lmsysorg/sglang:hy4-preview. Recommended sampling in the model card is temperature 0.9 and top_p 1.0. Reasoning effort defaults to high (deep chain-of-thought). For short answers, pass a template flag equivalent to reasoning_effort = "no_think". Tool-call and reasoning parsers are named hy_v4 in the vLLM recipe, which is a hint that agent loops are a first-class target.

Tencent also says Hy4 preview took part in optimizing parts of its own training and inference stack, including operator fusion and communication work that raised end-to-end throughput by about 31.8% versus a baseline in their report. Treat that as an interesting research claim rather than a customer SLA, but it fits the broader industry push toward models that help debug their own systems.

Known limits (and why Tencent shipped anyway)

Tencent is unusually blunt about preview issues. Reuters and the model card both call out two behaviors: the model can take longer than necessary on complex questions, and it can over-verify its own answers. Those are real costs in interactive coding agents and customer support flows. If you turn on high reasoning by default, budget latency and tokens accordingly, or route simple turns through the no-think path.

Hardware reality also bites. A 770B MoE at FP8 still wants a serious multi-GPU box. The published recipes assume eight-way tensor parallelism. That is fine for a lab or a well-funded platform team; it is not a laptop model. Most individuals will start on TokenHub or OpenRouter and only self-host if utilization and privacy justify the cluster.

Finally, “preview” means the team expects more pre-training and post-training headroom. Official Hy4 series follow-ups are already teased. If you pin an internal evaluation on today’s checkpoint, keep a re-run on the calendar when the next Hy4 drop lands.

Practical takeaways

• Evaluate for agentic coding first. The largest reported gains versus Hy3 are on SWE-style and terminal / tool benchmarks. Put it on a private suite of multi-file fixes, test writing, and refactor tasks before you trust marketing tables.
• Use the 1M context deliberately. Dumping an entire monorepo into the window is possible, but retrieval plus selective packing usually wins on cost. Save the full window for the cases where cross-file dependencies actually matter.
• Split reasoning modes. Keep high reasoning for planning, hard math, and ambiguous debugging. Force direct answers for classification, formatting, and short edits so latency stays predictable.
• Prefer FP8 + speculative decoding in production. Start from the official container recipes rather than reinventing quantization and MTP wiring. Measure acceptance rate and tokens/sec on your own prompts.
• Watch over-verification. If agents loop on “let me double-check,” add stop conditions, tighter tools, or a second-pass verifier model instead of letting Hy4 burn tokens proving the same claim.
• License is a feature. Apache 2.0 makes Hy4 preview a legitimate on-prem candidate next to other open MoEs. Confirm your counsel is comfortable with the Apache terms for your deployment, then compare total cost of ownership against closed APIs.
• Try hosted first, self-host later. Two weeks of free access on WorkBuddy / CodeBuddy (at launch) and paid API routes are enough to see whether quality fits your workload before you reserve eight GPUs.

Bottom line

Hy4 preview is not “just another 700B+ model.” It is an open, productivity-oriented MoE with a million-token window, native speculative decoding, and serving recipes that look written for people who actually run agents. Closed models may still lead on some hard reasoning suites, and the preview’s habit of over-thinking will frustrate anyone who bills by the second. But for teams that need strong open weights, long context, and a path from API trial to self-hosted deployment, Hy4 preview is one of the most consequential generative AI releases of late August 2026.

Sources: Tencent announcement, Hugging Face model card, Reuters, and independent model summaries such as AI/TLDR.