Any system that reads untrusted text while holding a production model key is a credential store with a chat interface. Anthropic's September threat report shows attackers have noticed, and are going after the eval harnesses and gateways that hold the keys rather than the models behind them.
In its 154-page report published September 10, Anthropic describes a group that planted malicious instructions in an AI vendor's automated evaluation sandbox, which then handed over production API keys for several model providers. Separately, it reports that multiple actors compromised AI wrapper services' implementation of LiteLLM and "used prompt injection to exfiltrate the production API keys used in their cloud-hosted container environments". Neither attack needed a flaw in a model. Both needed a key sitting next to text an attacker could write.
That raises a question most platform teams have never asked out loud: where does your production key live, and what reads untrusted input in the same process?
What the Attackers Took From the Eval Sandbox
The attackers took keys, not models — and then turned similar techniques on roughly 30 more AI companies. Anthropic tracks the group as GTG-50020, a Russian-speaking, financially motivated actor that had historically conducted intrusions against hotel booking and financial technology platforms. Its stated goal, pursued across more than a dozen avenues, was access to a pre-release Claude model. It never obtained one.
An evaluation sandbox is the environment that runs candidate models or agents against test tasks and scores what comes back. To do that job it needs live credentials for every provider it tests, and it has to ingest content — task files, model outputs, tool results — that nobody reviews line by line. Prompt injection is text crafted so that a model reading it treats it as instructions rather than data. Put those two things in the same runtime and you get what Anthropic describes: a sandbox that can be caused to "hand over the credentials it held".
The follow-on is the part to worry about. In a later campaign the group identified one successful attack path and repeated it against all thirty targets, "adapting slightly to account for differences across the targets" — roughly 30 AI companies in about four days. One working technique, industrialised inside a week.
Anthropic is explicit that the keys involved were customers' keys stolen from customers' environments, and that the actor never compromised Anthropic's own systems. The vendor whose sandbox gave up its keys is not named.
Why a Stolen Model Key Is Worth More Than Its Compute
A stolen model key pays an attacker three ways, which is why it has become a target in its own right. The report lists them as Loot, Compute and Cover: keys have resale value in established markets, attack workloads run at someone else's expense, and the activity is attributed to the credential's legitimate owner. Anthropic says access to AI through compromised API keys, session tokens and devices "has increasingly become the sole objective of multiple criminal groups".
Cover is the part your detection stack is least ready for. The report describes an operator that, on obtaining a target's API keys, "automatically switched to using the victim's keys instead of their own". From your provider's side, that traffic is you. It bills to your account, burns your rate limits, and any abuse investigation starts at your organisation.
The other cases in the report show how systematic the hunt has become:
- One pipeline mass-downloaded 1.8 million distinct Android APKs, decompiled them, and scanned for hardcoded secrets with TruffleHog.
- In a ShinyHunters-affiliated intrusion tracked as GTG-50014, one stolen API key was used for roughly three weeks to attack other organisations, including a French retail chain.
- GTG-50021 ran a fraudulent Claude reseller that silently proxied paying customers to a different model while a bundled client harvested their Anthropic credentials for resale.
The bill is not hypothetical. When Sysdig documented what it called LLMjacking in May 2024, it estimated the cost to a victim "can be over $46,000 per day" once an attacker maxed out quota on Claude models across regions.
Your LLM Gateway Holds Every Provider's Key at Once
Your LLM gateway is the most concentrated credential store in your AI stack, because holding every provider's key in one place is what it is for. That consolidation is the point — one control plane for routing, budgets and logging, which is why self-hosting one made sense in our LLM gateway buyer's guide. It is also what makes the gateway the one box worth attacking.
LiteLLM's 2026 advisories show how directly that plays out:
- CVE-2026-42208, a SQL injection in the proxy's API key verification path, affected versions 1.81.16 through 1.83.6 and was fixed in 1.83.7. Sysdig observed the first exploitation attempt roughly 36 hours after disclosure, aimed at the tables holding virtual keys, upstream provider credentials and proxy configuration.
- CVE-2026-42271, a command injection through MCP connectivity-test endpoints, let attackers read environment variables containing provider API keys injected at container startup. CISA added it to the Known Exploited Vulnerabilities catalog on June 8, 2026.
Neither CVE is prompt injection, and the September report does not say LiteLLM itself is flawed — it describes wrapper services' implementations being compromised. But the shape is the same in every case. The attacker only has to reach something that can read the process's environment or memory: an agent with a code-execution tool, a test endpoint that spawns a subprocess, a harness that runs model-written code.
Buying a hosted router such as OpenRouter or running Kong AI Gateway changes who operates that credential store, not whether one exists — the same dependency that came into focus when Stripe agreed to buy OpenRouter.
The Strongest Case That This Isn't Your Problem
The best counter-argument is that a model with no tools cannot leak a key it cannot see — and it is correct. A plain completion call that sends a prompt and gets text back has no way to read an environment variable. Prompt injection becomes credential theft only when the component reading hostile text also has a path to the secret: a shell, a file read, a subprocess, an outbound HTTP tool, or a log it can echo back.
The second counter-argument is that these were an AI vendor's sandbox and wrapper services, not an enterprise. Also fair. Now look at what you run. An eval harness scoring models before a switch — the kind of work behind sizing an eval set before you trust a gap. A shared gateway every team calls. Agent sandboxes that execute generated code.
Whether your evals run on a hosted platform such as Braintrust or Langfuse or in a homegrown harness, the test is identical: does anything that executes or reads untrusted content share a runtime with a production key? If the harness was built quickly by the team that already had production keys, check whether it borrowed them.
CI Pipelines Already Solved This in 2021
The software industry has solved this exact problem once, in continuous integration. In August 2021 GitHub's Security Lab warned that combining the pull_request_target trigger with a checkout of an untrusted pull request "is a dangerous practice that may lead to repository compromise" — because even a workflow that never references a secret still has the read/write repository token in memory, "potentially available to any running program".
The fix was not to scan pull requests harder. It was to split the job: an unprivileged workflow processes the untrusted code and stores its results as artifacts, then a separate privileged workflow acts on those artifacts with write permission. Untrusted input and credentials never share a runtime.
That maps onto evals almost one to one. The run that executes model outputs, agent tool calls and task files is the unprivileged job. Scoring, publishing, and anything that needs a real production key is a separate step that consumes the artifacts. Injection classifiers are a reasonable extra layer — but a filter you hope catches every malicious sentence is the 2021 equivalent of eyeballing pull requests for backdoors.
What to Do Before Your Next Eval Run
The immediate job is to find every place a production model key sits next to untrusted input, then make each of those keys small, separate and loud. Anthropic's own advice is to treat AI keys and agent integrations "with the same level of seriousness as they do production credentials". In practice that means:
This Week:
- List every runtime that holds a model key and reads content you did not write — eval harnesses, gateways, agent sandboxes, RAG ingestion workers, CI jobs that call models. For each, note whether a model or agent in it has a shell, file, subprocess or HTTP tool. Those are your credential stores with a prompt interface.
- Check your LiteLLM version against the advisories. Both CVEs affected releases up to 1.83.6 — but CVE-2026-59822, an MCP authentication bypass, affects every version below 1.84.0, and CISA has added it to its Known Exploited Vulnerabilities catalog, so upgrade to 1.84.0 or later. If an affected instance was reachable from an untrusted network, follow Sysdig's advice to rotate every virtual API key, master key and provider credential stored in it — not just the one you suspect.
- Find production keys in Anthropic's Default Workspace. You cannot set spend or rate limits on the Default Workspace, so a production key living there has no workspace-level ceiling at all.
This Month:
- Give evals their own keys inside their own boundary. Anthropic API keys can be scoped to a single workspace, and each workspace can carry its own monthly spend cap and rate limits. Archiving a workspace archives every API key created for it within seconds, and cannot be undone — a real break-glass switch, so rehearse who may pull it. Use the equivalent project or account boundary on every other provider. An eval key should be unable to serve production traffic.
- Split the harness like a CI pipeline. Run untrusted tasks and model-written code in a container that holds no provider credentials, and route its model calls through a gateway that attaches a short-lived, low-budget key on the server side. On LiteLLM, virtual keys support a per-key
modelsallowlist,max_budget,tpm_limitandrpm_limit, and adurationexpiry; regeneration with a grace period is an Enterprise feature, as is reading provider keys from Vault, AWS Secrets Manager or other secret managers. A secret manager keeps keys out of environment variables, which is exactly what CVE-2026-42271 read — but it does not protect them from code running inside the gateway: Wiz's honeypots caught attackers reading LiteLLM's master key straight out of the running process's Python module state. Budget for it, and still keep untrusted input away from the gateway process. - Alert on usage you did not schedule. D3 Security's reading of the report is blunt: "Put egress monitoring on your AI calls. Unexpected volume against a model endpoint is now a compromise signal." Break usage out per key or per workspace, baseline eval keys (bursty around scheduled runs, flat otherwise), and page someone when the pattern breaks. An attacker running on your key looks like you in every log except that one.
Before Your Next AI Procurement Cycle:
- Ban gray-market model access in writing. Anthropic says AI access should be purchased only through authorized channels; GTG-50021 is what a "discount" reseller can turn out to be. Put it in the acceptable-use policy, the procurement checklist and the expense policy.
- Ask every eval, observability and gateway vendor three questions: where your provider keys are stored, whether any component that processes model output or customer content can read them, and how fast they can revoke and rotate on your behalf. An unnamed vendor in this report is the reason to get the answers on paper.
- Scrub keys from traces and agent logs. Credentials also leak the slow way — through reasoning and trajectory logs that get shared or published.
The Bottom Line
This is the build-server lesson arriving in a new place. CI jobs spent years running untrusted pull-request code within reach of repository credentials, because the job needed both and splitting them was inconvenient. The documented fix was separation, not smarter scanning.
Eval harnesses and LLM gateways are at that point now, with one difference that makes it worse: the untrusted input is plain language, and the component reading it follows instructions for a living. You will not filter your way out of that. You can only make sure that when a model obeys the wrong sentence, nothing valuable is within reach.
Keep the key out of the room where the untrusted text gets read.