Every THE D[AI]LY BRIEF article on KV cache — enterprise AI analysis, benchmarks, vendor comparisons, and ROI frameworks for technology and business leaders. Updated as new coverage publishes.
Salesforce AI Research and UIUC replaced the scoring pass in KV cache eviction with a uniform random draw, kept the prompt pinned, and matched the strongest published evictor across four models and six reasoning tasks — at 32-43% higher vLLM throughput. Random-plus-prompt-protection is now the baseline any 'intelligent cache compression' claim has to beat.
September 9, 2026 · 13 min read
Tencent's Hy4 preview markets 49B active parameters, but all 770B must stay resident — roughly 780GB in FP8, which an 8xH100 node cannot load at all. Size a Mixture-of-Experts model on total parameters and price throughput on active ones.
August 30, 2026 · 12 min read
Nvidia's 3,400 tokens/sec on Groq 3 LPX and Cerebras' 4,400 on CS-4 are both single-stream figures. On a 256-LPU rack at 100K context, the memory math caps concurrency near a batch of 12 — so any capacity plan sized off a headline token rate is sized for one user.
August 29, 2026 · 12 min read