Google is quietly rebuilding its AI silicon supply chain. On April 20, 2026, The Information reported that Google is in active design talks with Marvell Technology for two new custom AI chips—a memory processing unit (MPU) that sits alongside existing TPUs, and a new TPU built specifically for inference. Marvell stock popped 6.3% in premarket and is now up roughly 84% on the year. Broadcom, Google's dominant TPU design partner with a contract running through 2031, slipped on the news. No contract is signed yet; design finalization is targeted for 2027.

Why this matters now: every enterprise already paying for AI has an inference cost problem that is about to get much worse. Training a model happens once. Inference runs every time a user asks a question, every time an agent takes an action, every time a customer clicks a personalized recommendation. Custom ASIC sales will grow 45% in 2026 according to TrendForce, and the market is on a path to $118 billion by 2033. For CIOs, CTOs, and CFOs still indexing their AI budgets to Nvidia H100/B200 pricing, the Google-Marvell deal is a preview of where hyperscaler unit economics are heading—and what that will mean for your cloud bill.

What Google Is Actually Designing

The Memory Processing Unit is the more interesting chip. Today's TPU and GPU architectures spend a huge fraction of their power budget shuttling data between compute and HBM memory. Every token generated during inference requires loading KV cache, activation tensors, and model weights from memory into compute, then writing results back. An MPU shortens that path by putting computation physically closer to memory—sometimes integrated into the memory stack itself—so the chip burns less energy moving bytes and more energy actually running the model.

The inference-optimized TPU is the volume play. Training chips are designed to maximize throughput on dense matrix multiplication at massive batch sizes. Inference workloads look nothing like that. They're sensitive to latency per token, memory bandwidth, and the ability to run many small concurrent requests without hot-spotting. An inference TPU lets Google match silicon to workload rather than pay the "training chip tax" on every user query. Marvell will operate in a design-services capacity, similar to MediaTek's role building the cost-optimized "e" variants of Google's Ironwood TPU at 20–30% lower cost than high-performance Broadcom versions.

2 million units is the scale number to anchor on. Reports cite a plan to manufacture roughly 2 million MPUs as part of this partnership. That's not a pilot. That's enough silicon to run a meaningful fraction of Google Cloud's inference traffic on a different cost curve than the Broadcom-designed TPUs dominating today.

TorchTPU is the software move nobody is paying enough attention to. Google has also shipped TorchTPU to make TPUs first-class citizens inside PyTorch, which removes one of the last structural reasons enterprise ML teams have defaulted to Nvidia. Until recently, the honest answer to "why are you on H100s?" was usually "because our code is written in PyTorch and CUDA Just Works." TorchTPU collapses that gap. If your team is writing inference services today, the framework lock-in argument against TPUs is weaker than it was even six months ago.

Why Inference Economics Matter More Than Training Economics

Training a frontier model is a one-time cost. Once it's done, it's done. Inference costs, by contrast, scale linearly with demand. Every enterprise copilot, every agent, every personalized recommendation is a persistent load on inference infrastructure. The CFO implication is stark: your training bill is a line item; your inference bill is a run-rate that grows with adoption.

The "efficiency tax" framing from the analyst community nails it. The cost of running AI is not just the chip price. It includes the overhead of stranded memory bandwidth, the power cost of data movement, the margin middlemen take at each layer of the stack, and the opportunity cost of not being able to match workload to silicon. Custom inference chips attack all four at once. A chip that spends 30% less power per token and 20% less die area on data movement doesn't just improve margins for Google—it reshapes what "cost per thousand tokens" means for every Google Cloud customer running Gemini, Vertex AI, or a hosted open model.

The Broadcom concentration risk is the other half of this story. Broadcom currently owns over 70% of the custom AI accelerator market, and its Google contract runs through 2031. That is extraordinary vendor concentration for a single hyperscaler. Adding Marvell as a third design partner alongside Broadcom and MediaTek doesn't eliminate that concentration—it de-risks it, while keeping enough Broadcom volume to honor the contract. For any CIO who has sat through a vendor concentration review, the pattern is familiar. You don't cut the incumbent. You bring in a competitor, you let the pricing team benchmark, and you quietly rebalance at each renewal cycle.

For CIOs: Four Implications for Your 2027 Cloud Strategy

1. Your Google Cloud inference pricing is going to diverge from your Nvidia-on-AWS pricing—and the gap is going to matter. Custom inference silicon lets Google price Gemini and Vertex workloads at margins Nvidia-based competitors cannot match. You should start modeling a world where the same quality of inference costs 20–40% less on Google Cloud than on an equivalent AWS or Azure deployment for certain workload profiles. That's not a 2030 forecast. It's what hyperscalers are explicitly designing for in 2026.

2. The inference workload taxonomy in your stack matters more than it did. Not all inference is equal. High-batch, latency-tolerant workloads (bulk embedding generation, background document analysis) benefit from very different silicon than low-latency interactive workloads (customer-facing chat, agent action execution). The vendors offering custom silicon will differentiate on workload profile. Start tagging your inference workloads now by latency sensitivity, context length, and burst pattern—so you can actually shop them when pricing diverges.

3. PyTorch portability just got real. TorchTPU plus ROCm plus CUDA means the software lock-in argument for staying on any single accelerator vendor is weaker than it has been since the early deep learning era. If your MLOps team still treats "we're a PyTorch shop" as a reason to default to Nvidia, that's now an outdated assumption. Validate portability on real workloads this year.

4. Power and sustainability math changes too. A significant fraction of enterprise AI TCO is now power, not just silicon. MPUs and inference-optimized TPUs promise meaningfully lower power per token. If you have internal ESG commitments or data center PUE constraints, custom silicon suddenly becomes a sustainability lever, not just a cost lever.

For CTOs: What to Pressure-Test in the Architecture

Don't take the MPU story at keynote rigor. Memory processing units are an architectural idea with a long history, limited production track record, and real software-stack implications. Ask your Google Cloud account team for concrete answers on three questions before you bet workloads on it:

• What programming model exposes the MPU to your code? Is it transparent at the XLA/JAX level, or does it require workload-specific rewrites? If it's the latter, the efficiency gains come at an engineering cost you need to price in.
• What's the real inference latency and throughput profile across context length? Memory-bound inference at 128K context looks very different from compute-bound inference at 4K context. The MPU's advantage will vary by workload; get benchmarks for your actual distribution, not marketing averages.
• How does the inference TPU compare to Nvidia B200/B300 on perf-per-dollar for your inference mix? The honest answer will be workload-dependent. Don't accept a single headline number.

The inference-chip-as-service pattern is what to watch for. Historically, hyperscalers offered custom silicon as an internal advantage and a generic managed service—"you get the benefit but you don't pick the chip." That's changing. Expect Google (and AWS with Trainium/Inferentia, and Microsoft with Maia) to expose SKU-level chip selection in the next 12–18 months, so customers can pick MPU-backed inference vs. Nvidia-backed inference explicitly. That puts procurement discipline back in your hands—and forces your platform team to have a real workload taxonomy.

The software stack is where this gets won or lost. Custom silicon is only as useful as the compiler, kernel library, and framework support behind it. Watch Google's investment in XLA, JAX, PyTorch/TorchTPU, and the ONNX ecosystem. If Marvell and Google ship silicon without matching compiler maturity, early adopters will eat the engineering cost. Let someone else be the reference customer—come in on the second cohort.

For CFOs: Three Numbers to Run Before the Next Renewal

1. Inference run-rate as a percentage of total AI spend. Most enterprises underestimate this. Pilots are training-heavy; production is inference-heavy. If inference is already >60% of your AI compute bill (and for most enterprises past pilot stage, it is), custom-silicon-driven pricing changes matter more to you than any model vendor discount.

2. Cloud concentration vs. workload portability. If 70% of your AI workloads run on one cloud, you have neither pricing leverage nor a credible threat. Modeling what it would cost to move 20% of inference workloads to a second cloud—even hypothetically—is the single best preparation for the pricing shifts custom silicon will unlock.

3. Power cost trajectory. Data center power costs are rising across most geographies. Custom silicon that reduces power per token is a direct CFO lever. Ask your Google/AWS/Azure AEs for power-per-million-tokens benchmarks on their custom-silicon SKUs vs. equivalent GPU SKUs. If they can't answer, that's signal.

The negotiating posture this deal enables is real. In 2024 and 2025, CIOs had to take hyperscaler inference pricing as given—Nvidia scarcity made every cloud a price-taker. In 2026, hyperscalers are building their own silicon at scale. That changes who is the price-taker and who is the price-setter. Walk into your 2027 renewal with a three-cloud inference benchmark, a clear workload taxonomy, and power-per-token data. You'll get materially better terms than the enterprise that walks in quoting list prices.

Competitive Read: Nvidia, Broadcom, AWS, and the Long Game

Nvidia is not in immediate trouble. Nvidia still dominates training, still owns the CUDA moat, and still has the best software stack in the industry. But the inference battle is shaping up differently. ASICs with 2–3x better perf-per-watt on specific inference workloads are real, and hyperscalers are the only customers large enough to justify the custom silicon engineering cost. Nvidia's counter is platform breadth and speed—its DGX Cloud, CUDA-X inference libraries, and the Rubin platform are built to keep the inference market from commoditizing too fast. Expect Jensen to aggressively price inference-specific SKUs and bundle software advantages that ASICs can't easily replicate.

Broadcom is the quiet winner either way. Even as Marvell enters the picture, Broadcom's 2031 contract and 70%+ market share make it the structural beneficiary of the custom ASIC boom. Broadcom's share dip on the Marvell news is a trading reaction, not a fundamental shift. The 45% ASIC market growth lifts every custom silicon vendor.

AWS is the other hyperscaler to watch. Trainium 2 and Inferentia 3 are AWS's answer to the same problem. Amazon has been quieter publicly but has deeper integration with enterprise customers than Google Cloud does. Expect AWS to respond to Google-Marvell with its own silicon design partner announcement within six months.

Microsoft Azure is the slow burn. Maia 100 is Azure's in-house chip, and the OpenAI + Anthropic relationship gives Microsoft unique leverage on inference demand. But Azure's custom silicon story is a quarter or two behind Google and AWS. Expect aggressive pricing on Azure AI Foundry to compensate while Maia ramps.

The Bottom Line

Google-Marvell is not a one-day stock story. It's the latest move in a three-year structural shift where inference—not training—becomes the center of gravity for AI infrastructure economics. Custom silicon from hyperscalers will compress inference pricing for workloads that fit the ASIC profile. Nvidia will respond with software and platform bundling. The enterprises that come out ahead are the ones that treat inference as a portable workload, not a lock-in, and negotiate accordingly.

The quiet takeaway for any CIO reading this: your cloud procurement posture needs to be built for a world where the chip under your inference workload changes every 18 months. That means workload taxonomy, portability testing, and power-per-token benchmarks become standard practice, not novelty. The vendors are already building for that world. Make sure your sourcing team is, too.

Don't over-rotate on the stock moves. Marvell is up 84% on the year because the market is pricing in a structural ASIC boom, not because of this specific deal. The operational question for your enterprise is narrower and more valuable: in 2027, when Google offers MPU-backed inference at a 25% discount to equivalent GPU capacity, will your architecture be portable enough to take the savings? That's the benchmark to plan toward. Everything else is market noise.

Want to calculate your own AI ROI? Try our AI ROI Calculator — takes 60 seconds and shows projected savings, payback period, and 3-year ROI.

Continue Reading

• Broadcom's $100 Billion AI Forecast and What It Means for Infrastructure
• SiFive's $400M RISC-V Bet Against Nvidia in the AI Data Center
• OpenAI's $20B Cerebras Deal: The Chip Diversification Play

Sources

• The Next Web: Google in talks with Marvell to build new AI inference chips alongside Broadcom TPU programme
• CNBC: Marvell pops on report it will help Google with custom AI chips; Broadcom shares sink
• The Information: Google in Talks With Marvell to Build New AI Chips for Inference
• Yahoo Finance: Marvell shares rally on report of new AI chip partnership with Google
• MediaPost: Google's Marvell Partnership Could Reduce 'Efficiency Tax'
• RS Web Sols: Google and Marvell Partner on Advanced TPU Designs for AI Inference