inference goodputWEKA and NAND Research Set an 8-Hour Inference Test
WEKA and NAND Research's Inference Economics Scorecard and SemiAnalysis's open AgentX traces give platform teams a way to test agent inference on eight-hour goodput. Here is what the vendor benchmark shows and where it falls short.
November 20, 2026 · 10 min readneocloud contractsYour Nvidia Neocloud Reservation Leaves Build Risk With You
Nvidia's $32B of neocloud commitments and $118B of guarantees take effect only after a site is financed, built and accepted. If you reserve GPUs in an unbuilt facility, the delay risk is yours to contract for.
November 18, 2026 · 9 min readexport controlsBIS Requires Licenses for AI Chips Shipped to Five Countries
A BIS interim final rule requires licenses for advanced AI chips going to Singapore, Malaysia, Indonesia, Thailand and Brazil, and for any data center giving China- or Macau-parented companies more than 12 million TPP of compute. US-parented customers are excluded only if the shipper files monthly reports.
November 16, 2026 · 10 min readKimi K3.1 ProMoonshot Makes Kimi K3.1 Pro Weights Free for Internal Use
Kimi K3.1 Pro's 1.56 TB weights are public and free for internal use. An eight-GPU B300 node breaks even with the $2/$10 API near 13 billion input tokens a month, and legal and tax scores fell from K3.
November 4, 2026 · 10 min readMistral Large 4Mistral Lets Companies Run Large 4 Internally for Free
Mistral Large 4's 1.05T-parameter weights are on Hugging Face. At FP8 they fit one eight-GPU DGX B200 node, internal use is free, and serving the model to outside customers needs a separate commercial licence from Mistral.
October 30, 2026 · 10 min readNVIDIA GTC Berlin 2026Nvidia Unveils DSX POD and OpenShell at GTC Berlin
Nvidia's DSX POD puts seven Vera Rubin racks in a truckable container, but partners ship it in early 2027 with no price. The open-source OpenShell agent runtime is the GTC Berlin announcement you can deploy this week.
October 22, 2026 · 9 min readIntel Crescent IslandIntel Says D-Series Runs Models up to 300B Parameters
Intel's air-cooled D-series GPU holds a 300B model on one PCIe card, but at 1.5 TB/s a large dense model streams single-digit tokens per second. Sort your workloads by capacity versus bandwidth before the quote arrives.
October 13, 2026 · 9 min readLLM quantizationFP8 vs NVFP4 vs INT4-AWQ: INT4 Beat NVFP4 on 3 of 4 Models
Run FP8 by default. For 4-bit, INT4-AWQ beat NVFP4 on speed on three of four models in an independent Blackwell benchmark, and NVIDIA's own NVFP4 Gemma file was nearly the size of FP8.
October 9, 2026 · 12 min readMistral Large 4Mistral Large 4 Preview Beats Kimi K3 on Legal Benchmark
Mistral Large 4 beats Kimi K3 on Harvey's legal agent benchmark and trails it on the Vals Index. It is a discounted preview API for now, and the licence for its end-of-October weights is unpublished.
October 6, 2026 · 9 min readGPU insuranceWhen the Colo's Chillers Fail, the GPU Loss Is Yours
S&P says there is no obvious insurance product for the GPUs inside AI data centers, and colo agreements often cap the operator's liability at three months of fees. If you own a GPU cluster in someone else's hall, the gap between that cap and the hardware is yours to close.
September 14, 2026 · 11 min readKV cacheRandom Eviction Matched the Scorers. Protect the Prompt.
Salesforce AI Research and UIUC replaced the scoring pass in KV cache eviction with a uniform random draw, kept the prompt pinned, and matched the strongest published evictor across four models and six reasoning tasks — at 32-43% higher vLLM throughput. Random-plus-prompt-protection is now the baseline any 'intelligent cache compression' claim has to beat.
September 9, 2026 · 13 min readMixture of ExpertsTencent Says 49B Active. Your Node Loads All 770B.
Tencent's Hy4 preview markets 49B active parameters, but all 770B must stay resident — roughly 780GB in FP8, which an 8xH100 node cannot load at all. Size a Mixture-of-Experts model on total parameters and price throughput on active ones.
August 30, 2026 · 12 min readAI inference3,400 Tokens/s Was Batch 1. At 100K Context, It's Batch 12.
Nvidia's 3,400 tokens/sec on Groq 3 LPX and Cerebras' 4,400 on CS-4 are both single-stream figures. On a 256-LPU rack at 100K context, the memory math caps concurrency near a batch of 12 — so any capacity plan sized off a headline token rate is sized for one user.
August 29, 2026 · 12 min readvLLMvLLM vs TensorRT-LLM vs SGLang: Default to vLLM
TensorRT-LLM 1.2 removed the TensorRT engine build entirely, invalidating most published advice on this choice. vLLM is the default, SGLang wins prefix-heavy agent traffic, and the properties that actually matter never appear in a benchmark.
August 24, 2026 · 18 min read