Inference Economics

OpenAI and Broadcom unveiled Jalapeño, a custom inference ASIC designed from scratch for LLM workloads. Built in nine months with AI-assisted design, it claims 50% lower cost per token than NVIDIA GPUs. With Google, Amazon, and Microsoft all building competing custom silicon, the era of GPU-only inference is ending — and the enterprise AI cost structure is about to be rewritten.

Cerebras serves a trillion-parameter open-weight model at 981 tokens/sec — 6.7x faster than GPU clouds. The inference cost math just changed for CIOs.