Random Eviction Matched the Scorers. Protect the Prompt.
Salesforce AI Research and UIUC replaced the scoring pass in KV cache eviction with a uniform random draw, kept the prompt pinned, and matched the strongest published evictor across four models and six reasoning tasks — at 32-43% higher vLLM throughput. Random-plus-prompt-protection is now the baseline any 'intelligent cache compression' claim has to beat.
September 9, 2026 · 13 min read