Most teams shopping for an OpenAI alternative are solving the wrong problem. They were told "the data can't leave the EU", they went looking for a European model, and they are about to spend two quarters migrating away from a constraint OpenAI already satisfies.
Here is the verdict, before the table. If your blocker is data residency, you almost certainly do not need to leave OpenAI: OpenAI's own documentation lists API residency in Europe (EEA plus Switzerland), the UK, Canada, Japan, India, Singapore, South Korea, Australia and the UAE, and Azure's Data Zone deployments keep processing inside the Azure EU Data Boundary. If your blocker is jurisdiction — a regulator, a customer contract or a board policy that says a US company must not be able to compel your data — then residency is irrelevant and only three of the seven routes below actually clear it. And if your blocker is vendor concentration, you need a second model family under a separate contract, which is a much cheaper project than either of the other two.
Every price below was read from the vendor's live page on 12 August 2026, normalised to one workload: 12 million tokens a day of document Q&A over customer records — roughly 10M in, 2M out — served from an EU legal entity, under a policy capping any single model vendor at 60% of production inference.
| Route | Deployment model | Clears | Does NOT clear | Price, checked 12 Aug 2026 |
|---|---|---|---|---|
| Azure OpenAI on Microsoft Foundry | Microsoft SaaS, EU/US/APAC data zone | Residency, procurement, SLA | US jurisdiction; model-vendor concentration | Per token; Data Zone billed separately from Global Standard |
| Google Gemini on Vertex AI | Google SaaS, EU multi-region endpoint | Residency including ML processing; second model family | US jurisdiction | Per token |
| Anthropic Claude on Amazon Bedrock | AWS SaaS, EU inference profile | Concentration; provider can't read your prompts | US jurisdiction; EU residency on Microsoft Foundry | Claude Sonnet 5 at $2.00 / $10.00 per 1M in/out; EU regional endpoints add a 10% premium |
| Mistral AI | French company, EU-hosted API, or self-deployed | US jurisdiction; residency; air-gap | Frontier reasoning gap vs GPT | Medium 3.5 $1.50 / $7.50; Small 4 $0.15 / $0.60 per 1M |
| Cohere North | Your VPC, on-prem, or air-gapped | Data never leaves your perimeter, with a vendor to sign for it | Published pricing; you supply GPUs | Model Vault from $4.00/hr ($2,500/mo); North: contact sales |
| IBM watsonx.ai + Granite | Cloud Pak for Data on OpenShift, disconnected install | On-prem with an accountable vendor; Apache-2.0 weights | Speed; Cloud Pak licensing complexity | Contact sales |
| Open weights on your own GPUs | Your hardware, vLLM | Everything | Your payroll | 8×H100 at $3.99/GPU-hr ≈ $31.92/hr ≈ $23,300/mo |
Why Data Residency Is Almost Never the Real Blocker
Residency is a solved problem at every major vendor, including the one you were told to leave. OpenAI documents regional data residency across ten geographies, and approved customers can enable Zero Data Retention, under which, on endpoints such as /v1/responses and /v1/chat/completions, "the store parameter will always be treated as false, even if the request attempts to set the value to true". Microsoft's Data Zone Standard deployments process prompts and responses only within the specified zone, where the EU zone follows the Azure EU Data Boundary and can extend to EFTA states such as Norway and Switzerland. Google goes furthest on paper: it commits to "performing ML processing of your data within the same specific region or multi-region where it's stored", explicitly including "inference and all other ML processing that happens prior to output generation" — a stronger statement than storage residency, because inference is where your data is actually exposed. That commitment is what you get on Vertex AI; it is not what you get from a model vendor's own endpoint by default.
Two caveats before you treat that as settled, because both bite in procurement. Residency is not a checkbox: OpenAI's own guidance is to "Contact our sales team to see if you're eligible for using data residency controls," and any region other than the United States additionally requires approval for abuse monitoring controls and a signed Modified Retention amendment. And the guarantee is not uniform across those ten geographies — coverage is per-endpoint and varies by region, with Australia, for example, documented for storage but not processing. Confirm the region on your blocker actually covers inference, not just storage, before you tell anyone the problem is solved.
So if your legal team wrote "EU data residency" on the blocker, take it back to them and ask which of three things they meant.
There is one genuine residency trap in OpenAI's stack, and it is not the one people worry about. The endpoints that are ineligible for Zero Data Retention are the stateful ones, including /v1/vector_stores, /v1/files, /v1/batches, /v1/evals, /v1/fine_tuning/jobs, /v1/assistants and /v1/threads. If you built a retrieval application on OpenAI's managed vector store and file search, your inference is covered and your corpus is not. That is the finding worth escalating — and it is an architecture problem, not a vendor problem. Moving your index onto infrastructure you control fixes it without changing model vendors at all.
What a Jurisdiction Rule Actually Requires
A jurisdiction rule is cleared by the nationality of the entity that can be served a legal order, not by the location of the disk. In June 2025, Microsoft's director of public and legal affairs in France told a Senate inquiry under oath that the company could not guarantee French citizens' data held under public contracts would be withheld from US authorities without French consent. He noted no such demand had been made, and that Microsoft contractually commits to resisting unfounded requests. Both things are true at once: the controls are real, and the guarantee does not exist.
That single admission is why the first three rows of the table fail a strict sovereignty test no matter how good their residency documentation is. Azure, Vertex and Bedrock are US-controlled entities. A Data Zone deployment moves your bytes; it does not move the subpoena.
Three routes survive:
Mistral AI is a French company, so a US order has no direct purchase on it. Mistral's published API pricing puts Medium 3.5 at $1.50 per 1M input and $7.50 per 1M output tokens, and Small 4 at $0.15 / $0.60. On our 12M-token workload, Medium 3.5 lands at roughly $30 a day — about $900 a month — and Small 4 at roughly $81 a month. Mistral also documents self-deployment and recommends vLLM, "a highly-optimized Python-only serving framework which can expose an OpenAI-compatible API". That last clause is the migration story in one line: the SDK swap is nearly free.
Cohere sells the shape most regulated buyers actually want — a supported product that runs inside your walls. Its private deployment documentation is specific in a way marketing pages are not: Embed and Rerank models need 1 GPU, Command models need 2, and "all images in the proxy.replicated.com registry are available to pull and copy into an air-gapped environment". Access is gated by a Cohere licence ID that doubles as your registry credential. Cohere's pricing page publishes Model Vault rates — Embed 4 Small at $4.00/hr or $2,500/mo, Rerank 4 Pro Large at $10.00/hr or $6,500/mo — but North itself is contact-sales, which you should treat as a real cost signal, not a formality.
Open weights on hardware you own is the only route with no counterparty at all. Qwen 3 ships under Apache 2.0 at most sizes; IBM's Granite 4.1 family, released 29 April 2026, is Apache 2.0 across language, vision, speech, embedding and Guardian models and is "optimized for popular open-source inference runtimes, including vLLM, SGLang, and llama.cpp". Meta's Llama is the outlier: the Llama 4 Community License requires a separate licence from Meta above 700 million monthly active users, and obliges you to prefix any derivative model's name with "Llama" and display "Built with Llama". Those terms are survivable for most enterprises and are not open source. Read them before your legal team does.
Where Each Option Breaks — Including the One I'd Skip
Say who should not buy each of these and the comparison starts being worth something.
Do not buy Azure OpenAI Data Zone as a sovereignty answer. It is an excellent residency and procurement answer — the same GPT models on Microsoft paper via Azure AI Foundry, with EU processing — and it is the cheapest way to satisfy a residency-only blocker. It does nothing about the CLOUD Act, and selling it internally as "sovereign" will cost you credibility the first time your risk committee reads the French Senate transcript.
Do not route Claude through Microsoft Foundry if you are European. This is the specific loser in this comparison, and it is a routing choice rather than a vendor judgement. Anthropic's models on Foundry are deployed in Sweden but run as Global Standard, which by Microsoft's own definition means the service "can process prompts and responses in any geography where the model is deployed". InfoQ reported in July 2026 that no European data zone exists for Claude on Foundry, with Anthropic's compliance page listing it as "Coming 2026" and no date attached — and a Dutch bank refusing the route on exactly that basis. If you want Claude with EU processing today, take it through Amazon Bedrock's EU cross-region inference profiles instead, where, per AWS's June 2026 guidance, "requests from a source Region that lies in the EU can only be routed to other AWS Regions with the European Union."
Bedrock has a second property worth knowing, and it answers the question executives actually ask — can the model company read our prompts? AWS documents that each model provider's software is deep-copied into a Model Deployment Account owned by the Bedrock service team, and "because the model providers don't have access to those accounts, they don't have access to Amazon Bedrock logs or to customer prompts and completions." Anthropic's own commercial terms say plainly that "Anthropic may not train models on Customer Content from Services," that the customer "owns its Outputs," and that Anthropic will defend customers against third-party IP claims arising from authorised use — with carve-outs for customer modifications and customer-supplied inputs.
Do not buy Mistral expecting frontier parity. You are trading the top of the reasoning curve for a jurisdiction you can defend. On document Q&A, classification, extraction and summarisation that trade is usually invisible. On long-horizon agentic work it is not, and you will find out during evaluation rather than during procurement.
Do not buy Cohere North if you cannot supply and operate GPUs. Two GPUs per Command model is modest, but it is still a platform team, a capacity plan and an upgrade path you now own.
The one I would skip for most buyers is IBM watsonx.ai — unless you already run Cloud Pak for Data on OpenShift, in which case it is the obvious answer and you should stop reading. Red Hat documents that OpenShift AI "can easily be deployed on-prem in an air gapped or disconnected configuration," and Granite's Apache 2.0 licence is genuinely permissive. But everything about watsonx is priced through sales, the platform arrives as a Cloud Pak entitlement rather than a model endpoint, and if the goal is air-gapped inference on open weights, you can pull Granite from Hugging Face and serve it on vLLM without buying the platform around it. Buy watsonx for the governance and lifecycle tooling, not to escape OpenAI.
Self-Hosting Is a Control Decision, Not a Cost Decision
Run the arithmetic before you write the business case, because at enterprise-typical volumes self-hosting costs more, not less. Lambda publishes on-demand H100 SXM at $3.99 per GPU-hour in an 8-GPU configuration — $31.92 an hour, about $23,300 a month for one node running continuously, before storage, egress, or a single hour of engineering time.
Our 12M-token workload costs roughly $900 a month on Mistral Medium 3.5. Self-hosting is therefore about 26× more expensive at that volume. Blending Mistral's published input and output rates at our 5:1 ratio gives about $2.50 per 1M tokens, so one node only breaks even somewhere north of 300 million tokens a day — roughly 3,500 tokens per second sustained. That throughput is not the obstacle: published vLLM benchmarks put a single H100 at 3,600-5,500 output tokens a second on 7-9B models, so an eight-GPU node clears 3,500 comfortably on mid-size open weights. The obstacle is keeping it saturated around the clock. Break-even is a duty-cycle problem, not a throughput one, and enterprise document Q&A is exactly the workload that idles overnight.
That is not an argument against self-hosting. It is an argument against justifying it on price. Self-host when the requirement is no third party may hold this data under any circumstance — and put the $23,300 a month in the compliance budget line where it belongs, not the infrastructure savings line where it will be challenged every quarter.
The Migration Bill Nobody Quotes You
The SDK swap is the cheap part; the expensive part is everything stateful you built on top of OpenAI. Mistral, vLLM and most self-hosted stacks expose an OpenAI-compatible API, so the client library change is typically a day.
What actually costs money, in descending order:
- Your retrieval layer, if it lives in OpenAI's managed primitives.
/v1/vector_stores,/v1/filesand/v1/assistantshave no drop-in equivalent anywhere else. You are rebuilding, not porting — and note those are the same endpoints excluded from ZDR, so this work is on your critical path either way. - Re-running every eval. A model swap invalidates your entire quality baseline. If you do not have evals, this line item is "build evals," and it is the real reason migrations slip.
- Tool-calling and structured-output behaviour. Schemas transfer; reliability does not. Budget for prompt rework on every agentic path.
- Prompt-caching economics. Cache hit rates and discounts differ per vendor, and a workload tuned around one vendor's caching can get materially more expensive on another even at a lower headline rate.
- Two sets of everything in governance. A second model family means a second DPA, a second sub-processor list, a second set of eval evidence and a second entry in the risk register.
The Decision, in Four Questions
Ask these in order and stop at the first "yes":
- Can a US legal order reach this data without our consent, and is that unacceptable? Yes → Mistral, Cohere private deployment, or open weights on your own hardware. Nothing else qualifies.
- Must the data stay inside our own network perimeter? Yes → Cohere North, watsonx on OpenShift, or self-hosted open weights. Rule out all four SaaS routes.
- Is the rule about concentration rather than location? Yes → add a second model family under a separate contract. Claude on Bedrock or Gemini on Vertex is a weeks-long project, not a quarters-long one.
- Is it residency only? Then stay where you are, switch to a Data Zone or regional endpoint, and move your vector store off the managed service. You are done.
What changes the answer: a merger that puts you under a new regulator, a customer contract with a sovereignty clause you did not negotiate, or a model release that closes the capability gap you cited last year. Re-run question 1 at every renewal.
What to Do in the Next 30 Days
This Week: Get the blocker in writing. Ask legal for the exact clause — statute, contract or internal policy — and classify it as residency, jurisdiction or concentration. Most of this decision is made or unmade in that one email.
This Month: Audit which OpenAI endpoints you use, specifically flagging /v1/vector_stores, /v1/files and /v1/assistants. Then run your top three production prompts against Mistral Medium 3.5 and one Apache-2.0 open-weight model, scoring them on your own eval set — not a public benchmark.
Before Your Next Renewal: Price the winner at three volumes — today's, 3×, and 10× — and put the self-hosted node's $23,300 a month next to it. Take both numbers to the risk committee together, so the trade is decided as a control decision with a price tag rather than a cost decision with a compliance story attached.
The Bottom Line
This is the mainframe-to-cloud argument running backwards, and the same rule applies: the organisations that got it right were the ones that named the constraint precisely before they named the vendor. Everyone else bought a migration to solve a memo.
Seven routes, three real answers, and one question that decides between them. If a US company can be compelled to hand over the data and that is genuinely unacceptable, you have three options and they all cost more than you want. If it is not, you have a configuration change and a sales call.
Don't buy sovereignty when you were only asked for a region.
Continue Reading
- Claude vs GPT vs Gemini: Stop Comparing Per-Token Prices
- Your Assistants API Dies Aug 26. Azure's Exit Is Different.
- Best RAG Platforms for Regulated Industries: Permissions First
- DeepSeek Will Raise Prices. Your Ceiling Is Already 4x.
- Best LLM Gateways for Cost Control: Self-Host First
- EU AI Act Governance Tools: Buy Inventory, Not Policy Packs
- Britain Regulated Four Clouds. Not the Models Inside.
- Free 550B Model: NVIDIA Ends Self-Hosted AI Quality Gap
