vLLM

Salesforce AI Research and UIUC replaced the scoring pass in KV cache eviction with a uniform random draw, kept the prompt pinned, and matched the strongest published evictor across four models and six reasoning tasks — at 32-43% higher vLLM throughput. Random-plus-prompt-protection is now the baseline any 'intelligent cache compression' claim has to beat.

Tencent's Hy4 preview markets 49B active parameters, but all 770B must stay resident — roughly 780GB in FP8, which an 8xH100 node cannot load at all. Size a Mixture-of-Experts model on total parameters and price throughput on active ones.

Nvidia's reported $12.9B deal for Hugging Face puts optimum-neuron, optimum-habana, optimum-intel and optimum-amd — the Transformers bridge for every rival accelerator — inside Nvidia. optimum-tpu was archived read-only in January, and its own README points the exit at vLLM.

A new arXiv paper measures the corrective value of a failed tool call in an agent transcript and finds it negative on every small model tested: repeat probability rises from 0.06 to 0.54. Clean-context retry was the worst harness tested; paraphrasing the failure removed 76% of the effect for free.

TensorRT-LLM 1.2 removed the TensorRT engine build entirely, invalidating most published advice on this choice. vLLM is the default, SGLang wins prefix-heavy agent traffic, and the properties that actually matter never appear in a benchmark.