A Mixture-of-Experts model has two parameter counts, and they size two different things. The active count tells you what a token costs to compute. The total count tells you how much GPU memory you have to buy before you serve a single request. Vendors lead with the flattering one.
Tencent released and open-sourced Hy4 preview on 28 August 2026 with "770B total parameters and 49B active parameters" and a context window "exceeding 1M tokens," under Apache 2.0. It is one of the largest permissively-licensed flagships anyone has shipped. Somebody on your team has already opened a build-vs-buy spreadsheet and typed 49 into the parameter cell. That number is wrong for every line below it except throughput.
The 49B Is a Compute Spec, Not a Memory Spec
Activation sparsity reduces the arithmetic per token. It does not reduce the bytes you keep resident. The Hy4-preview model card describes 78 layers — the first a dense FFN, the remaining 77 Mixture-of-Experts — where "every token activates the top-8 routed experts along with the shared expert" out of 256 routed experts.
Do the arithmetic from the published config. Each expert is three matrices of hidden_size 6144 by moe_intermediate_size 2048 — about 37.7 million parameters. Multiply by 256 experts across 77 MoE layers and you get roughly 744 billion parameters that exist purely as experts. Nine of them fire per token per layer. The other 248 sit in memory doing nothing, because the router picks a different eight for the next token.
That is the whole trick, and the whole trap. Which experts activate is decided per token, per layer, at inference time. You cannot predict the set, so you cannot prefetch it without stalling the forward pass. Every expert has to be addressable at every step, which in a configuration that serves production traffic means every expert has to be resident. Expert parallelism and tensor parallelism change which GPU holds which bytes; neither changes how many bytes there are.
At FP8, one byte per parameter, that is 770GB of weights, plus another 10GB for the native multi-token-prediction layer the GitHub repository lists at "10B total parameters, 0.7B activated." Call it 780GB before a single token of context exists.
And that 780GB is the floor, not the default. The FP8 number requires a separately-published checkpoint, tencent/Hy4-preview-FP8. The headline repo everyone will git clone declares torch_dtype: "bfloat16" with no quantization config at all — Hugging Face reports it at 780B params in BF16, which is roughly 1.56TB of weights, double the figure above. Both of Tencent's own launch commands serve the FP8 repo, not the one the announcement links to. Check which artifact your capacity plan is actually sizing.
Where 780GB Fits, and Where It Does Not
The cheapest 8-GPU node that can load this model is not the one most enterprises already have reserved. Run the capacities:
| Node | Memory per GPU | 8-GPU total | Fits 780GB FP8? | Fits ~1.56TB BF16? |
|---|---|---|---|---|
| 8x H100 | 80GB | 640GB | No | No |
| 8x H200 | 141GB | 1,128GB | Yes, 348GB spare | No |
| 8x B200 | 192GB | 1,536GB | Yes, 756GB spare | No, short before KV cache |
| 8x B300 | 288GB | 2,304GB | Yes, 1,524GB spare | Yes, 744GB spare |
An 8x H100 node — the workhorse configuration of the last two years — tops out at 640GB and cannot hold the FP8 checkpoint at all, let alone the BF16 one.
The honest caveat: "cannot hold" means cannot hold in HBM. Expert-offload runtimes genuinely do run 671B-class MoE models on a fraction of that memory by parking experts in system RAM — KTransformers reports DeepSeek-R1 on a single 24GB GPU, and llama.cpp has MoE-aware CPU offload — exploiting the fact that per-token expert work is a small batched matrix-vector product a CPU can absorb. That is a real technique and it is not a serving configuration: it trades an order of magnitude of throughput for the memory saving. If someone counters your capacity plan with "but it runs on one GPU," this is what they mean, and it does not survive contact with a concurrency target.
This is where the deployment recipe matters more than the press release. Tencent's model card ships a vLLM command with --tensor-parallel-size 8 and an SGLang command with --tp-size 8, which reads like "eight GPUs, any eight." The vLLM recipe page for Hy4-preview is more specific about what was actually tested: "16xB200, 8xB300." Sixteen of Nvidia's 192GB parts, or eight of the 288GB parts. Not H200s, and emphatically not H100s.
Line that up against the BF16 checkpoint and the sixteen stops looking arbitrary: 1.56TB does not fit in an 8x B200 node's 1,536GB, and 8x B300 at 2,304GB is the smallest eight-way box that takes it. The tested configurations are the ones that hold the weights Tencent actually published, which is not the number in the announcement.
Note also that --tensor-parallel-size 8 is a sharding instruction, not a hardware requirement. It says split every tensor eight ways. It says nothing about whether those eight cards have 640GB or 2.3TB between them. A capacity plan built by reading that flag is a capacity plan built on a coincidence.
The 1M Context Is Paid For in KV Cache
Long context is a second memory bill, and it lands after the weights. Hy4 uses Gated DeepSeek Sparse Attention over a compressed latent cache — kv_lora_rank 512 plus qk_rope_head_dim 64, so 576 values cached per token per layer. Across 78 layers that is 44,928 values per token — and the cache does not inherit the weights' FP8. vLLM's kv_cache_dtype defaults to auto, which its documentation defines as "will use model data type": bfloat16 here, two bytes per value, even when the weights are quantized. Neither Tencent's published command nor the vLLM recipe passes the flag that would change it. So the number to plan with is 87.8 KiB per token, or about 94GB of KV cache for one sequence at the full 1,048,576-token context. Setting --kv-cache-dtype fp8 halves it to 43.9 KiB, but that is an opt-in no quickstart hands you.
The word "sparse" invites the wrong conclusion here. As Sebastian Raschka puts it, MLA "compresses what is stored in the KV cache," while DSA "changes which stored positions the attention operation reads." Sparse attention cuts the compute and the bandwidth of reading the cache. It does not shrink the cache. You still store every token you have seen.
Now put both bills on one node. vLLM's gpu_memory_utilization defaults to 0.92, so on 8x H200 you have about 1,038GB to work with. Subtract the 780GB FP8 checkpoint — the BF16 one never fits here — and roughly 258GB remains for KV cache and activations. At 94GB per full-context sequence that is two concurrent 1M-token requests — before you account for activation workspace. Drop to a 32K working context and the same headroom carries around 87 concurrent sessions.
That is the number your capacity model needs, and neither "49B" nor "1M" is it. We made the same point about vendor throughput benchmarks in 3,400 Tokens/s Was Batch 1. At 100K Context, It's Batch 12 — the headline figure is measured in a regime nobody serves production traffic in.
Worth noting: even Tencent's own API does not sell you the round million. Its documentation specifies a "1M context window, 960k maximum input, and 64k maximum output."
What the Node Costs Against a $0.834 API
At published rates, the hosted API is cheaper than a dedicated node until you are consuming tens of billions of tokens a month. Tencent prices Hy4 preview at US$0.834 per million input tokens, US$2.501 per million output, and US$0.042 per million on cache hits — figures confirmed on both the Tencent Cloud FAQ and the OpenRouter listing, which also reports about 41 tokens/sec and 3.13s P50 latency.
Against that, the rental side. GMI Cloud's provider comparison puts CoreWeave's H200 at $6.31 per GPU-hour sold only as an 8-GPU bundle, priced as of May 2026 — about $50.48/hour, or $36,850 a month running continuously. Thunder Compute's B300 survey, dated 28 August 2026, lists five providers from Modal at $7.10 per GPU-hour to AWS at $17.80, a median of $7.89 — so the vendor-tested 8x B300 configuration is roughly $63/hour, or $46,000 a month.
Convert that to tokens at Tencent's rates. The $36,850 H200 node equals about 44 billion input tokens or 14.7 billion output tokens a month. Sustained around the clock, 14.7 billion output tokens is roughly 5,600 output tokens per second, every second, for thirty days.
Ask your team a plain question: is our traffic flat? Because the break-even assumes it is. An enterprise workload with a European morning peak and a dead weekend does not run a rented node at 100% — and the node bills at 100% regardless. This is the same utilization trap that governs GPU cloud procurement generally and that we walked through on cost per token across accelerators.
The honest steel-man for self-hosting: if you already own the silicon, marginal cost per token collapses and this analysis inverts. Rented capacity and owned capacity are different arguments. Do not let a vendor deck blur them — and note that server memory pricing is moving against anyone building new.
The Reason to Self-Host Anyway Is Not Cost
One provider serves this model, and it is Tencent Cloud. OpenRouter lists Hy4 preview as "hosted by one provider." Tencent Cloud offers a Singapore endpoint with "Global" resource scope and a Silicon Valley endpoint at tokenhub-us.tencentcloudmaas.com scoped to the United States — and its own FAQ cautions that "Global" scheduling "does not mean unconditional sale in every country, absence of network restrictions, or automatic compliance with local data rules."
If your data classification policy will not route regulated content to a Chinese vendor's endpoint, the API is not a cheaper option. It is not an option. Self-hosting is the only path, and the $46,000-a-month node is the price of the policy, not the price of the tokens. That is a legitimate reason to buy the hardware. "It says 49B" is not.
Two related disciplines apply the moment you take custody of open weights. Mirror the checkpoint rather than pulling it live at deploy time, per the weight-mirroring playbook. And if you are consuming this model through a vendor's product rather than directly, read the model card they are actually running — provenance is now a procurement question, not a curiosity.
Do This Before You Approve the GPU Reservation
This Week:
- Grep every capacity model and vendor deck in flight for a Mixture-of-Experts parameter count. If the cell holds the active number, replace it with the total and re-run. The ratio here is 15.7x.
- Write down your P95 working context, not your maximum. Multiply by your target concurrency, then by 87.8 KiB per token for this model — the BF16 default, not the FP8 best case. That product is your KV cache line, and it competes with the weights for the same HBM.
- Confirm which GPU SKU your reservation actually delivers, and which checkpoint your team intends to pull. H100 versus H200 is not a performance tier for this model — it is load versus will-not-load. FP8 versus BF16 is the same question asked about the artifact instead of the hardware, and the answer moves the requirement by 780GB.
Before You Sign the Reservation:
- Model both cost curves side by side: rented node at 100% billing versus API at your real diurnal token profile. If the crossover sits above your forecast, buy the API and revisit in two quarters.
- Get your data classification decision in writing first. If regulated data cannot reach a Tencent Cloud endpoint, the cost comparison is moot and you are buying hardware for a compliance requirement. Price it as such.
This Quarter:
- Benchmark on the runtime you will actually operate, not the one in the README. Our runtime comparison still lands on vLLM as the default, and the Hy4 recipe ships a purpose-built vLLM image with a
FLASHMLA_SPARSEattention backend. - Re-price your existing open-weight commitments. Model prices move fast — DeepSeek's own API repricing is the reminder that a hosted rate is a decision someone can change, while resident memory is physics.
The Bottom Line
This is not new, it is just bigger. DeepSeek-V3 shipped "671B total parameters with 37B activated for each token" in its model card, and the industry spent a year arguing about a 37B model that needed a rack. Hy4 raises the total to 770B at a ratio of nearly 16 to 1 — DeepSeek-V3's was over 18 to 1. The ratio is not what keeps climbing; the resident total is, and that is the number on the invoice.
There is a real gift here. Apache 2.0 weights on a frontier-class model, downloadable from Hugging Face, served through OpenRouter, at under a dollar per million input tokens — that is a genuine shift in what a mid-sized enterprise can reach, and it belongs in the same conversation as the broader move to Chinese open-weight models. TechNode reported Tencent's internal blind evaluation of 163 experts across 203 engineering tasks scoring it 2.99 out of 4.00, against GLM-5.3 at 2.92 and Kimi K3 at 2.94 — a vendor's own number, and a narrow margin, but not a trivial model.
Open weights and affordable weights are different properties. Size the model on what must be resident. Price it on what actually fires.
Continue Reading
3,400 Tokens/s Was Batch 1. At 100K Context, It's Batch 12. vLLM vs TensorRT-LLM vs SGLang: Default to vLLM GPU Clouds Compared: Nebius on Price, CoreWeave at 3AM AMD's Inference Discount Depends on a GPU You Can't Rent Hugging Face Hired Bankers. Go Mirror Your Weights. DeepSeek Will Raise Prices. Your Ceiling Is Already 4x.