Buy the runtime your cloud contract already covers, keep the agent graph in code you compile, and weight exit cost as the single heaviest line on your scorecard. That is the recommendation, and the reason is not philosophical. In the twelve months to August 2026, four of the products that would have topped a 2025 agent-platform shortlist either shut down, were superseded, or changed the unit they bill you in. The feature grid you are about to build will be stale before your pilot finishes. The exit column will not be.
Here is the landscape mapped to the criteria that actually predict regret.
| Platform | Where the graph lives | Runtime metering | Documented rollout limit | Per-agent identity | Contract change in last 12 months |
|---|---|---|---|---|---|
| Amazon Bedrock AgentCore | Your container, any framework | $0.0895/vCPU-hr + $0.00945/GB-hr | None documented | IAM + AgentCore Identity | None (GA Oct 2025) |
| Microsoft Foundry Agent Service | Your container, any framework | vCPU-hr + GiB-hr, 2 vCPU ceiling | No traffic splitting between versions | Dedicated Entra ID per agent | Semantic Kernel unified into Agent Framework |
| Google Gemini Enterprise Agent Platform | Your code — ADK, LangGraph, LlamaIndex, custom | vCPU-hr + GiB-hr | None documented | Google Cloud IAM | Renamed from Vertex AI; session/memory metering added Jan 2026 |
| LangGraph + LangSmith Deployment | Your code, LangGraph only | $0.0675/vCPU-hr, plus a database line at $0.177/vCPU-hr | None documented | You build it | Repriced July 2026 |
| Temporal Cloud | Your code, any framework | $50/million actions, $100/mo floor | None documented | You build it | None |
| CrewAI AMP | CrewAI only | Not published | Not published | Not published | Two tiers only: free or custom |
| OpenAI Agent Builder | Vendor canvas | N/A | N/A | N/A | Shuts down Nov 30, 2026 |
Notice what the runtime metering column does not show: a differentiator. Three of these products bill agent compute in the same two units, and the published rates sit inside a 25% band. That is the first thing a selection framework has to absorb — the line procurement spends three weeks negotiating is the one where the market has already converged.
Four Contract Changes in Twelve Months
An agent orchestration platform is the layer that decides which model call happens next, holds the state between those calls, and retries when one fails. It is the most consequential thing you will buy and the least stable.
The record, from the vendors' own deprecation and pricing pages:
OpenAI's Assistants API shuts down on August 26, 2026, exactly a year after the sunset was announced, pushing customers onto the Responses and Conversations APIs. Then, on June 3, 2026, OpenAI announced that Agent Builder and the Evals platform close on November 30, 2026 — Evals goes read-only a month earlier, on October 31. Agent Builder shipped in AgentKit in October 2025. It got thirteen months. The migration path is the Agents SDK or ChatGPT Workspace Agents, and the workflow graphs you drew in the canvas do not convert into SDK code automatically.
LangChain changed its billing unit. LangSmith Deployment now meters in LangChain Compute Units and Storage Units — $1.50 per LCU, $1.00 per LSU — replacing the previous per-run and uptime charges. Per LangChain's own documentation, the new model applies to new usage from July 15, 2026, and organisations already on the old model transition on October 1, 2026. That is a fair, well-signposted change with real grandfathering. It is also a repricing that lands inside a normal enterprise budget cycle, on a line item nobody modelled.
Microsoft folded two frameworks into one. Agent Framework 1.0 reached GA on April 3, 2026, unifying "the enterprise-ready foundations of Semantic Kernel with the innovative orchestrations of AutoGen into a single, open-source SDK" and shipping with migration assistants that "analyze your existing code and generate step-by-step migration plans." Microsoft is doing this the right way — stable APIs, backward-compatibility commitment, tooling. It is still a rewrite on somebody's roadmap.
None of these are scandals. Every one was announced in advance with a documented path. That is precisely the point: even well-behaved vendors change the contract — four of the seven products in the table above did so inside twelve months, so a scorecard that does not price migration is measuring the wrong thing.
The failure rate underneath all this is real. Gartner predicted in June 2025 that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls — and estimated that only about 130 of the thousands of self-described agentic vendors are real. A framework's job is to survive being wrong about the vendor.
The Six Criteria, and What They Weigh
A scored rubric is only defensible if the weights are argued. These are mine, with the argument attached. Score each 1-5 and multiply.
Exit cost — 30%
The question: can one engineer move this workload to a different runtime inside one sprint? Not "is there an export button." Can they do it.
The discriminator is where the graph lives. If your control flow is Python or C# that you compile, test and commit, the runtime is a container host and swapping it is a deployment change. If your control flow is a diagram in the vendor's canvas, it is not source code, it is configuration in someone else's database — which is exactly why OpenAI's Agent Builder graphs do not convert to Agents SDK code.
This gets 30% because it is the only criterion that pays off in every scenario where you were wrong about the other five.
Durable execution and version rollout — 20%
The question: what happens when the host process dies 40 minutes into a 90-minute run, and can you ship a new version to 5% of sessions first?
Durable execution means the orchestrator persists each completed step so a crash resumes rather than restarts. Temporal built a business on this; LangGraph checkpoints state; the hyperscaler runtimes vary. Test it by killing the process, not by reading the datasheet.
Version rollout is where a specific, checkable answer hides. Microsoft's documentation for Foundry hosted agents states plainly that an agent endpoint "serves one version at a time and routes 100% of its traffic to that version" and that "Traffic splitting between versions isn't supported". If your release process depends on canarying a prompt change to a fraction of traffic, that is a hard architectural fact you will discover in month four unless you score it in week one.
Billing-unit stability — 15%
The question: has the vendor changed what it meters — not the price, the unit — in the last 24 months?
A price rise is a negotiation. A unit change is a re-forecast, because your consumption model was built on the old unit and there is no conversion factor. LangChain's move from per-run and uptime charges to LCU/LSU is the live example. Ask every vendor directly, in writing, and put the answer in the file.
Identity boundary — 15%
The question: does each agent get its own principal, and is authorisation enforced at the tool call rather than in the prompt?
Uber's engineering team wrote up why they rebuilt their identity stack for agents: their identity model was built around humans and workloads rather than an entity "authorized to act for or in the place of another," and execution context — the originating user, the intermediate agents — "is dropped across agent hops," which breaks both audit and fine-grained policy. Their answer was an agent registry, a security token service that embeds the fully attested actor chain into short-lived single-hop tokens, and an MCP gateway that enforces authorisation at tool invocation. They report P99 token-exchange latency under 40 milliseconds across thousands of internal agents.
That is the reference architecture. Now score your candidates against it. Foundry gives every hosted agent its own dedicated Microsoft Entra ID, created automatically at deploy time, which is the strongest out-of-the-box answer on this criterion. On the open frameworks, you are building Uber's stack yourself.
Evidence export — 10%
The question: can you get raw execution traces out, in a schema that will still parse in 2031?
If you deploy a high-risk system in the EU, Article 12 of the AI Act requires that the system "technically allow for the automatic recording of events (logs) over the lifetime of the system," and Article 19 requires providers to keep those logs "for a period appropriate to the intended purpose... of at least six months." Those obligations no longer bind from August 2, 2026: the AI Omnibus regulation entered into force on July 27, 2026, deferring high-risk requirements to December 2, 2027 for standalone Annex III systems and August 2, 2028 for AI embedded in regulated products. The extra runway does not change the retention design.
The trap is assuming "supports OpenTelemetry" solves this. It does not yet. As of mid-2026 the GenAI semantic conventions have moved to their own repository and every gen_ai.* span, metric, event and attribute still carries Development status — none is marked Stable. A vendor's OTel export is real and useful; it is not a stable schema you can build a six-year retention policy on. Score whether you can export the raw payload, not whether the logo appears on the slide.
Framework neutrality — 10%
The question: does the runtime genuinely host the framework you chose, and can you prove it in an afternoon?
All three hyperscalers claim neutrality and mostly mean it. AWS states AgentCore "works with any open source framework (CrewAI, LangGraph, LlamaIndex, Google ADK, OpenAI Agents SDK) and any model in or outside Amazon Bedrock". Microsoft's hosted agents documentation says you can use "any agent framework — the protocol libraries are framework-agnostic," naming Agent Framework, LangGraph, Semantic Kernel and custom code. Google's Agent Platform lists ADK, LangGraph, LangChain, LlamaIndex, Agent2Agent, AG2 and custom agents — notably not CrewAI.
Read those lists. If the framework your team already knows is missing from one, that is a real constraint hiding behind a neutral-sounding marketing line.
This weighs only 10% because the answer is mostly yes, and because the interoperability floor keeps rising underneath it: the A2A protocol passed 150 supporting organisations and reached v1.0 under Linux Foundation governance in April 2026, with integrations in Azure AI Foundry, Copilot Studio and AgentCore Runtime.
Why the Rate Card Is Not on the List
Normalise the three published runtimes to the same workload — three always-on replicas at 2 vCPU and 4 GiB each, which is 4,380 vCPU-hours and 8,760 GiB-hours a month — and the spread collapses.
AWS publishes AgentCore Runtime at $0.0895 per vCPU-hour and $0.00945 per GB-hour, which is $474.79 a month for that shape. LangSmith Deployment meters runtime compute at 0.045 LCU per vCPU-hour and memory at 0.006 LCU per GiB-hour; at $1.50 per LCU that is $0.0675 and $0.009 respectively — $374.49 a month, and a 25% cheaper headline rate. Microsoft and Google meter agent runtime in the same two units — Foundry Agent Service charges nothing extra to run a native agent and bills hosted agents by vCPU-hour and GiB-hour on top of model tokens.
Then read the next two lines on LangChain's page. Deployment also meters database compute at 0.177 LSU per vCPU-hour and database memory at 0.025 LSU per GiB-hour. At $1.00 per LSU, the database costs 2.6x more per vCPU-hour than the agent it serves. Add a single always-on 1-vCPU, 4-GiB database and that $374.49 becomes roughly $577 — about 21% more expensive than AgentCore, from the option with the cheaper compute rate. Seats are extra at $39.
And all of it is noise. A production agent doing meaningful work spends its money on model tokens; the orchestration runtime is a few hundred dollars a month against an inference bill one or two orders of magnitude larger. Do not build a selection framework around the rate card. Build it around the fact that AWS's pricing page breaks AgentCore into thirteen separate components — Runtime, Gateway, Memory, Identity, Browser, Code Interpreter, Web Search, Observability, Evaluations, Optimization, Policy, Agent Registry and Payments, eleven of them carrying an AWS rate of their own — that Microsoft's hosted-agent billing is "based on cpu + memory consumed across all active sessions, so oversizing multiplies cost by your concurrency", and that CrewAI publishes exactly two tiers — a free Basic plan with 50 workflow executions a month, and Enterprise on request. What you are scoring is whether the bill is forecastable, not whether it is low.
The Proof of Concept: Six Weeks, One Graph, Two Runtimes
Most agent PoCs prove the wrong thing. They prove a model can do the task, which you already knew, and they prove nothing about the platform, which is what you are buying.
Design it the other way round.
Week 0 — write the eval set before the code. Fifty to two hundred labelled cases from real historical work, with the correct outcome for each. Use the failure taxonomy from the Berkeley MAST study, which annotated over 1,600 execution traces across seven multi-agent frameworks and derived 14 failure modes in three categories — system design issues, inter-agent misalignment, and task verification — at an inter-annotator agreement of κ = 0.88. Those three categories are your failure buckets. If you cannot write the eval set, you do not understand the workflow well enough to automate it, and that is a finding worth six weeks on its own.
Weeks 1–2 — build the graph once, in code, on runtime A. One workflow. Real data. Real permissions. Not a demo tenant.
Week 3 — deploy the same artifact to runtime B. This is the criterion that carries 30% of the score, so it gets a week of the pilot rather than a paragraph in the RFP. Do not build a second agent. Take the container or the module you already have and stand it up somewhere else. The number of engineer-days this takes is your exit cost, measured rather than asserted. Under three days and portability is real. Over two weeks and you are buying a platform, not renting a runtime — which may still be the right call, but now you know the price.
Week 4 — break it on purpose. Kill the host process mid-run. Revoke a downstream token while a tool call is in flight. Return malformed JSON from a tool. Feed it a prompt-injection payload through a document it retrieves. Record what the platform does, not what the docs say it does.
Week 5 — cost and evidence. Run the eval set at ten times pilot volume and read the actual bill by line item. Export the traces and try to load them into your SIEM or warehouse. If the spans only render inside the vendor's UI, you have an evidence-export problem, and you found it in week five instead of during an audit.
Week 6 — write the exit plan. Not a slide. A document naming the artifacts you would move, the services you would have to rebuild, the data you would need to extract, and the engineer-days from week three. If nobody will sign it, that is the answer.
Two Sets of Exit Criteria, and You Need Both
"Exit criteria" means two different things here and teams routinely define only the first.
Kill criteria for the pilot — thresholds agreed in writing before week one, so the decision is arithmetic rather than politics:
- Task success on the held-out eval set below the manual baseline, with no credible path to closing the gap.
- Portability test in week three exceeding ten engineer-days.
- Cost per completed task above the fully loaded human cost at projected volume.
- Any unrecoverable state after a mid-run crash.
- Any tool invocation the platform cannot attribute to a specific agent identity and originating user.
Contract exit criteria — what goes in the agreement, because the pilot ends and the negotiation begins:
- A billing-unit change clause: if the vendor changes the metering unit, you get a stated notice period and the right to terminate without penalty. LangChain volunteered roughly eleven weeks between its new model going live on July 15 and the forced transition on October 1. Write it down anyway.
- Data and artifact export on termination, with a format specified and a deadline, covering execution traces and agent definitions — not just your documents.
- Change-of-control notification, which in this market is not theoretical.
- A deprecation notice minimum for any API your agents call. OpenAI gave twelve months on the Assistants API and about six on Agent Builder. Six months is a quarter of engineering planning; ask for twelve.
The Loser Is the Canvas
Every product in the table above has a defensible use. One category does not: the vendor-hosted visual builder as the system of record for production agent logic.
The case against it is not aesthetic. A drag-and-drop graph is genuinely faster for the first two weeks and genuinely better for letting a non-engineer express intent. The problem is what it is made of. It is a row in the vendor's database with no representation in your version control, no diff in your pull requests, no unit test, and no meaning outside that vendor's runtime. When the product sunsets, there is no migration — there is a screenshot and a rewrite. OpenAI's Agent Builder is the proof: shipped October 2025, wind-down announced June 2026, dead November 2026, graphs that do not convert.
Use a canvas for prototyping and stakeholder alignment, then rewrite the winner in code before it touches production traffic. The canvas is the on-ramp, not the road — and the test of whether your organisation understands that is whether anything drawn in one is currently serving customers.
And say who should not buy each of the rest, because a comparison where everything wins is not a comparison:
- Skip AgentCore if you are not already on AWS, or if you need one forecastable number — its pricing page breaks the service into thirteen components across eleven separate rates, and observability lands on your CloudWatch bill rather than the AgentCore one.
- Skip Foundry hosted agents if you need more than 2 vCPU per session, or if your release process requires canarying a version to a slice of traffic. Both are documented limits, not gaps.
- Skip Google's Agent Platform if your teams have standardised on CrewAI, which is absent from its supported-framework list.
- Skip LangSmith Deployment if a mid-contract change to the metering unit would break your forecast, and read the database line before you model the bill.
- Skip Temporal if you want anything AI-specific in the box. It is a durable execution engine: the Essentials plan carries a $100 monthly minimum with 1 million actions included, and the next five million cost $50 per million before the volume tiers start biting. You build the agent parts. Buy it when the workflow, not the model, is the hard problem.
- Skip CrewAI AMP if procurement requires a published rate before a pilot. Two tiers, one free and one on request, is not a posture most enterprise procurement functions can process.
What to Do in the Next 90 Days
This week. Write down which of the six criteria your current shortlist has scored. If exit cost is not on it, the shortlist is a feature comparison wearing a suit. Then ask each vendor, in writing: have you changed your metering unit in the last 24 months, and what notice will you give before the next change?
This month. Pick one workflow and write its eval set — before you pick a platform. Fifty labelled cases is enough to start. Separately, get an answer to the identity question: can each agent hold its own principal today, or is that a project?
Before quarter close. Run the six-week pilot with the week-three portability test in it, and produce the exit document. The deliverable of a platform evaluation is not a recommendation. It is a measured number for how much it costs to be wrong.
The Bottom Line
Every technology cycle has a layer that looks like the strategic choice and turns out to be the commodity. In the 2010s it was the container runtime: teams ran year-long bake-offs, and the answer became Kubernetes underneath and nobody's differentiator. Agent orchestration is following the same curve, faster. The A2A protocol went from launch to v1.0 with 150 organisations in twelve months. Three hyperscalers converged on the same billing unit, the same framework-agnostic container contract, and the same agent-to-agent protocol, all inside a year.
What is not converging is which of these products exists in 2028. So stop trying to pick the winner. Pick the arrangement where being wrong costs a sprint: the runtime comes from the cloud contract you have already signed, the graph lives in a repository you own, the identity boundary is enforced at the tool call, and the exit cost is a number somebody measured in week three of the pilot.
Score the exit. The features will change anyway.
Continue Reading
- LangChain Alternatives: Fix Durability Before You Rewrite
- Your Assistants API Dies Aug 26. Azure's Exit Is Different.
- AWS Agents Run 14 Days. The Session Is the Only Wall.
- Best AI Agent Monitoring: Langfuse, Then a Real Kill Switch
- Agentic AI Pricing: Don't Buy Consumption Without a Cap
- One Agent Escalated Another. Every Call Was Authorized.
- The Build-to-Buy Spectrum: 5 Tiers for Enterprise AI Agents
- EU AI Act Governance Tools: Buy Inventory, Not Policy Packs