Etched Sohu
by Etched
Transformer-only inference ASIC shipped as rack-scale frontier inference clusters
Etched Sohu is a transformer-only inference ASIC delivered as complete liquid-cooled racks, aimed at organisations whose unit economics are dominated by the cost and latency of serving large language models. By hard-wiring attention into fixed-function silicon instead of running it on general-purpose GPUs, it targets far higher throughput per watt than an NVIDIA HGX system — at the price of running nothing but transformers.
Etched is a San Jose chip company founded in 2022 by Gavin Uberti and Chris Zhu that builds Sohu, an ASIC purpose-built for transformer inference, and ships it as rack-scale Frontier Inference Clusters rather than as a standalone accelerator card. Where a GPU executes attention as programmable matrix-multiply instructions, Sohu burns the transformer architecture into fixed-function silicon, trading generality for throughput, latency and power efficiency. The chip is manufactured on TSMC's N4P 4nm process — Etched reports first-pass success with its A0 silicon — and carries 144GB of HBM3E per chip at roughly 1.8x the memory bandwidth of an H100 SXM5. Two proprietary techniques anchor the design: Low-Voltage Inference, which runs the math blocks at under half the voltage of most AI chips so racks sustain 80%+ of peak FLOPs without thermal throttling, and Cluster-Scale Memory, a shared low-latency HBM/SRAM pool stitched together by Etched's own ultra-low-latency interconnect so that, per the company, trillion-parameter sparse mixture-of-experts models, long context and agentic workloads stay resident across chips. Etched came out of stealth in mid-2026 with more than $1B in signed customer contracts, and on 18 August 2026 announced a $700M round at a $21B valuation led by Jane Street — roughly double the $10.3B mark set in July — bringing total capital raised to nearly $2B. Jane Street is also the only publicly confirmed customer to have taken delivery of a rack and is deploying it into live quantitative-trading workloads. Etched employs about 400 people, roughly 15% of them former NVIDIA engineers.
The VP of Infrastructure or Head of AI Platform at an organisation serving one or a few fixed transformer models at enormous, sustained volume, where cost-per-token and tail latency — not model variety — determine the P&L.
Dramatically lower cost and power per token for transformer inference than a general-purpose GPU rack, delivered as a turnkey liquid-cooled system rather than a component integration project.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Contact for pricing
- Target Market
- CTOs, VPs of Infrastructure, Enterprise Developers, ML Platform Engineers
- Deployment
- Self-hosted
- Founded
- 2022
- Headquarters
- San Jose, California, United States
- Team Size
- 201-500
- Customers
- Not disclosed; Jane Street is the only publicly confirmed customer, against more than $1B in signed contracts
Key Features
- ✓Transformer-only ASIC
Implements attention as fixed-function silicon rather than programmable matrix-multiply instructions, removing kernel-launch overhead and the generality that costs power.
- ✓Low-Voltage Inference (LVI)
Runs the math blocks at under half the voltage of most AI chips, so racks hold 80%+ of peak FLOPs without thermal throttling.
- ✓Cluster-Scale Memory (CSM)
A shared low-latency HBM/SRAM memory pool across chips over a proprietary interconnect, so very large models stay resident cluster-wide.
- ✓144GB HBM3E per chip
About 1.8x the memory bandwidth of an H100 SXM5, which matters because transformer decoding is memory-bound rather than compute-bound.
- ✓Frontier Inference Clusters
Etched co-designs chip, board, cooling plates, interconnect and rack, so the customer buys a working system rather than integrating parts.
- ✓TSMC N4P first-pass silicon
The A0 stepping taped out successfully on a 4nm process, removing one of the largest schedule risks facing a first-time ASIC vendor.
Use Cases
- •Low-latency trading inference
Jane Street took delivery of the first rack and is deploying it into live quantitative-trading workloads where response time is revenue.
- •High-volume LLM serving
Inference providers running a stable transformer model around the clock can cut cost per million tokens against GPU racks.
- •Power-constrained data-centre expansion
Sites limited by megawatts rather than capital can raise served tokens per rack without adding grid capacity.
- •Agentic and long-context workloads
Cluster-Scale Memory keeps long contexts and large sparse mixture-of-experts models resident across chips during multi-step agent runs.
- •Dedicated capacity for a frontier model
Organisations standardising on one model for years can amortise specialised silicon against a predictable, unchanging serving profile.
Ideal For
Best For
- ✓Latency-critical inference in quantitative trading, where Jane Street is the first confirmed production deployment
- ✓Inference providers serving a stable set of transformer LLMs at very high sustained token volume
- ✓Operators whose data-centre economics are constrained by power and rack density rather than capital cost
- ✓Large-scale agentic serving where long context and high tokens-per-second dominate the workload
- ✓Teams already committed to a fixed model architecture for the multi-year life of a hardware purchase
Not Ideal For
- ✗Anyone who needs to train or fine-tune models — Sohu is an inference-only part with no training path
- ✗Teams running diffusion, vision-language, state-space or other non-transformer architectures, which a transformer ASIC by definition cannot execute
- ✗Buyers who need a supported migration from vLLM or TensorRT-LLM, since serving runs on Etched's proprietary compiler stack
- ✗Organisations that require independent third-party benchmarks before committing capital — none have been published
Deployment
Market & Ratings
Not disclosed; Jane Street is the only publicly confirmed customer, against more than $1B in signed contracts
Market Analysis
Pros
- ✓Architectural specialisation attacks the real bottleneck: transformer decoding is memory-bound, and Sohu pairs 144GB HBM3E with roughly 1.8x an H100's bandwidth
- ✓First-pass TSMC N4P silicon and a shipped rack put it materially ahead of most challenger-silicon startups
- ✓Rack-scale co-design removes the integration burden that has slowed adoption of alternative AI accelerators
- ✓Over $1B of contracted demand before first shipment, and a customer using it in production rather than a lab
Cons
- ✗No independent third-party benchmarks exist — the headline throughput figures are vendor-reported and untested at production batch sizes
- ✗Architecture obsolescence is the core risk practitioners raise: a shift toward state-space models, Mamba, diffusion or other non-transformer designs would strand the hardware
- ✗Independent analysis of the original Sohu design argued it could not run vision-language, mixture-of-experts, diffusion or state-space models at all; Etched's 2026 rack marketing now claims trillion-parameter sparse MoE support, and nothing public resolves the contradiction
- ✗Serving requires Etched's proprietary compiler with no migration path from vLLM or TensorRT-LLM, creating hard lock-in
- ✗Inference only — no training or fine-tuning — and a single publicly confirmed customer means supply, support and roadmap risk are all concentrated
Pricing
Frontier Inference Cluster
Contact for pricing
- ✓Rack-scale liquid-cooled system with Sohu ASICs
- ✓Proprietary ultra-low-latency interconnect and Cluster-Scale Memory
- ✓Etched compiler and serving stack
No list pricing is published anywhere. Etched sells whole racks under negotiated multi-year supply contracts — it disclosed more than $1B in signed customer contracts before shipping its first system — so buyers should expect capital procurement with delivery-schedule terms, not an hourly or per-token rate. There is no cloud or on-demand option to trial the hardware.
Security & Compliance
Connect
Sources
This page was written from 4 sources, 3 on domains other than etched.com.
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
Gimlet Cloud
Multi-silicon inference cloud that splits AI agent workloads across GPUs, CPUs and accelerators
DigitalOcean Managed Agents
Managed agent runtime with microVM sandboxes, 16,000+ governed tools and serverless inference, billed on active CPU
Modular
MAX inference framework and Mojo language for serving AI models on NVIDIA, AMD and other chips
ZML/LLMD
Free, Python-free LLM inference server that runs open models on NVIDIA, AMD, Google TPU, Intel and Apple chips from one binary