In April, Meta built a leaderboard that ranked its 85,000+ employees by how many AI tokens they consumed. Engineers competed for titles like "Token Legend" and "Cache Wizard." The leaderboard came down two days after the story became public. Amazon ran a similar experiment — it lasted eight weeks before token inflation made the data meaningless. Both companies didn't stop measuring AI. They stopped measuring the wrong thing.
The story of these two leaderboards is the most instructive enterprise AI lesson of 2026. Not because the companies embarrassed themselves — they didn't, they corrected quickly — but because the mistake they made is one that most enterprises are still making right now, quietly, inside their own dashboards.
Measuring token consumption and calling it AI productivity is the enterprise equivalent of measuring keystrokes and calling it software quality. You get a number that goes up reliably every quarter. You get nothing useful about whether the organization is getting better.
The Anatomy of Both Failures
Meta's internal leaderboard, which employees nicknamed Claudeonomics after Anthropic's Claude model, was not an official management tool. An employee built it on the company intranet using internal usage data, and it ranked the top 250 token consumers across the company. The moment it surfaced publicly — reported by The Information in April — it came down. Meta said the employee removed it at their own discretion. Whether or not that is precisely how it happened, the implicit message from leadership was clear.
Amazon's version, called KiroRank, was tied to the company's Kiro developer platform. It ranked employees by AI activity with a target of more than 80% of developers using AI each week. According to the Financial Times, staff quickly discovered the incentive structure and began assigning AI agents to unnecessary tasks purely to climb the rankings. Engineers gave the practice a name: tokenmaxxing. The bill followed. Amazon shut down KiroRank on May 29, and a senior vice president told staff directly: do not use AI just for the sake of using it.
In both cases, the same thing happened. The organization created a metric that was easy to instrument and rewarded behavior that looked like AI adoption. Employees responded rationally to the incentive. The metric went up. The value did not.
Why This Mistake Spreads Upward
Here is the part that most enterprises miss. The token-counting failure does not stay in the engineering leaderboard. It travels upward.
When a CIO reports to the board on AI progress, the natural temptation is to reach for numbers that move — API calls, model invocations, active users, sessions per month. These numbers are already in the billing data. They cost nothing to instrument. And they go up every quarter with no intervention required.
The problem is what they do not measure. They measure what employees spent on AI. They do not measure what the organization got.
A McKinsey survey published in February, covering 330 companies across Southeast Asia and benchmarked against global averages, put a number on the gap. Around six in ten respondents reported less than 5% EBIT impact from their AI use. Close to one in five reported no discernible financial effect at all. Unclear ROI ranked among the top barriers to value capture, alongside talent shortages and integration complexity.
That is not a Southeast Asia problem. That is a measurement problem with a Southeast Asian data point. The same pattern shows up in every major enterprise AI survey from the past 12 months. McKinsey, Gartner, Deloitte, and PwC all report some version of the same finding: investment is rising, token usage is rising, and bottom-line impact is lagging significantly.
The China Counter-Example Clarifies the Issue
Understanding why token consumption is seductive as a metric requires stepping back to see it at macro scale. At the China Development Forum in Beijing earlier this year, National Data Administration director Liu Liehong announced that China's daily AI token usage had reached 140 trillion — up from roughly 100 billion at the start of 2024. He gave tokens an official government name, ciyuan, and described them as the settlement unit connecting technological supply with commercial demand. China is now reporting token throughput at policy forums the way other governments report industrial output.
That is a defensible use of the number. At a national economy level, token throughput tells you something real about how much inference capacity is being utilized across the country. It is a reasonable proxy for technological adoption at scale.
The error is not in the metric. It is in the level at which it is applied. When the same number travels from national economic policy down to an individual engineer's performance review, it loses everything that made it meaningful. The economy cares about aggregate throughput. The enterprise cares about margin, revenue, and competitive position. Those are different questions requiring different answers.
What Separates High Performers
The McKinsey data does more than document the measurement gap. It identifies what the companies successfully creating EBIT impact actually do differently.
High performers are roughly twice as likely to fundamentally redesign workflows rather than layer AI onto existing processes. They invest at a different magnitude — more than a third put over 20% of their digital budgets into AI, compared to the majority who allocate 11–20%. They formalize governance and define when human validation is required.
The common thread is that they treat AI as an operational capability, not a tooling upgrade. They ask: what business outcome changes when AI is involved here? Then they measure that outcome. Not the tool usage. The outcome.
In customer service, that means measuring cost per resolved ticket, not AI-assisted sessions. In finance, it means measuring hours redirected from reconciliation to analysis, not prompts submitted. In supply chain, it means measuring forecast accuracy improvement, not model invocations.
This sounds obvious. It is not easy. Outcome metrics require workflow redesign. They require knowing what "good" looks like before AI arrived so you can measure the delta. They require accountability for results rather than activity.
For CTOs: What the Evidence Says to Measure
If you are responsible for AI infrastructure and engineering productivity, the Amazon and Meta stories offer a direct lesson. The metrics that survived contact with reality were normalized deployments — Amazon's replacement for KiroRank — which count AI-assisted code that actually reaches production. Not code generated. Code shipped.
That distinction matters. An engineer who generates 50,000 tokens of AI-written code and ships none of it has created no value. An engineer who generates 2,000 tokens and ships a feature that reduces incident response time by 30% has created measurable value. Only one of those shows up meaningfully on a productivity metric.
The metrics worth tracking technically are: AI-assisted pull requests merged, incident response time with and without AI tooling, code review cycle time, and production defect rates for AI-assisted vs. manually written code. These require more instrumentation than a billing dashboard. They are also the only numbers that will withstand a CFO asking whether the AI investment is working.
For CFOs: The Accountability Framework AI Spending Actually Needs
From a finance perspective, AI spending currently occupies an awkward category. It is too large to treat as a rounding error and too diffuse to evaluate with project-level ROI analysis. The token-counting instinct is understandable from a budget management perspective — you want a unit of consumption you can price and track.
The more useful framing is to treat AI like any other operational input: labor, equipment, or infrastructure. You would not measure operational efficiency by counting how many hours employees worked or how many electricity kilowatt-hours you consumed. You would measure output per unit of input.
For AI, that means defining the output before deploying the tool. In a sales context, that might be average deal cycle length or proposal turnaround time. In legal, it might be contract review hours per engagement. In operations, it might be exception rate in automated processing workflows.
PwC's research suggests agentic AI can redirect up to 60% of finance team time from process execution to insight work. That is a testable hypothesis. You can measure where finance team hours go today, deploy AI into specific workflows, and measure whether the ratio shifts. That is a CFO-grade measurement framework. "We consumed 2.3 million tokens in Q2" is not.
The Metric Choice Is a Strategic Decision
There is a governance dimension to this that most enterprises are underweighting. How you measure AI determines what behavior you incentivize, which determines what kind of AI culture you build.
Organizations that measure token consumption build cultures of AI usage. Engineers learn to use AI frequently, document that usage, and report high engagement numbers. Organizations that measure outcomes build cultures of AI accountability. Engineers learn to ask which problems AI actually solves, instrument the before-and-after, and defend the investment.
Only one of those cultures generates sustainable ROI.
Meta and Amazon both corrected quickly — within weeks of the leaderboard stories breaking. They moved to metrics tied to production output. The fact that they tried token-counting and discarded it is not embarrassing. It is evidence that even sophisticated organizations with mature AI programs had to learn this lesson through experience.
Most enterprises are at an earlier stage than Meta or Amazon. They have not yet built the leaderboards. They are still debating what to put on the dashboard. The lesson from both companies is that this decision matters and needs to happen deliberately, before the incentive structures calcify.
The question for every enterprise AI leader right now is not how many tokens your organization consumed last quarter. It is what changed in the business because of it.
If you cannot answer that question cleanly, you are counting the wrong thing.
Measuring AI ROI in your organization? I'd hear how other enterprise leaders are approaching this — connect on LinkedIn or follow the conversation on X.
