SambaNova
by SambaNova Systems
Fast, low-cost inference for open models on custom dataflow chips, available in the cloud or in your own data center.
SambaNova builds its own AI inference chips (the Reconfigurable Dataflow Unit) and the full stack around them. It sells that stack three ways: SambaCloud, a pay-per-token API for open models; SambaStack, a hosted or on-premises inference platform; and SambaRack systems. It is aimed at enterprises, neoclouds and sovereign-AI operators who need fast, cheap inference for open-weight models without depending on GPUs.
SambaNova Systems, founded in 2017, designs the Reconfigurable Dataflow Unit (RDU), a custom AI chip with a three-tier memory hierarchy, and sells it as a vertically integrated inference stack. In April 2025 the company laid off 77 people, about 15% of a roughly 500-person workforce, and stopped focusing on training to concentrate on inference and cloud delivery of open-source models. The product line now has three parts. SambaCloud is a pay-per-token API that currently serves MiniMax-M2.7, DeepSeek-V3.1, Meta-Llama-3.3-70B-Instruct and gpt-oss-120b in production, with MiniMax-M3, DeepSeek-V3.2 and Google gemma-4-31B-it in preview. SambaStack is a turnkey inference platform that runs either hosted (SambaNova operates the hardware and Kubernetes) or on-premises (the customer controls everything), with OIDC/LDAP authentication and built-in prompt caching. SambaRack SN40-16 and the fifth-generation SN50 are the rack systems. SambaNova says the SN50 supports up to 2TB of DDR5, 520MB of SRAM and 64GB of HBM3 per chip, scales to 256 sockets sharing one memory space, and is due to ship in the second half of 2026, with SoftBank as its first deployer. Reported acquisition talks with Intel (about $1.6B) stalled in early 2026. Instead, SambaNova raised a $350M Series E in February 2026, with Intel Capital as a strategic investor in a multi-year partnership, and then a $1B Series F first close at an $11B valuation in July 2026, led by General Atlantic. JPMorganChase has selected it to run on-premises inference. The company competes with Groq, Cerebras and GPU-based inference clouds on tokens per second and cost per token.
Platform or AI infrastructure leads at enterprises, neoclouds or sovereign-cloud operators who serve open-weight LLMs at high volume and want an alternative to GPUs, either as an API or as on-premises racks.
High-throughput, low-latency inference of large open models such as DeepSeek, MiniMax, Llama and gpt-oss, at a lower cost per token, without having to build a GPU serving stack.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Usage-based, Freemium, Contact for pricing
- Target Market
- CTOs, CIOs, AI Platform Teams, Enterprise Developers, Cloud and Sovereign-AI Operators
- Deployment
- API-based, Hybrid, Self-hosted
- Founded
- 2017
- Team Size
- 201-500
Key Features
- ✓Reconfigurable Dataflow Unit (RDU) chips
A custom AI accelerator with a three-tier SRAM/HBM/DDR memory hierarchy, designed to hold very large models and decode fast without GPUs.
- ✓SambaCloud inference API
Pay-per-token hosted access to open models such as DeepSeek-V3.1, MiniMax-M2.7, Llama 3.3 70B and gpt-oss-120b, with a free tier for evaluation.
- ✓SambaStack hosted or on-prem platform
A turnkey inference platform that runs either managed by SambaNova or fully on-premises, with OIDC, LDAP or Keycloak authentication and access control by user group.
- ✓Built-in prompt caching
SambaStack's prompt caching reaches cache hit rates above 90%, and SambaCloud bills cached input on MiniMax models at a tenth of the normal input price ($0.06 vs $0.60 per million tokens).
- ✓SN50 rack-scale systems
The fifth-generation SambaRack is built for agentic inference, scaling to 256 sockets that share one memory space so very large or multiple models can be served together.
- ✓Broad framework and tool integrations
Documented integrations with LangChain, LlamaIndex, CrewAI, LangGraph, LiteLLM, n8n, Cursor, Claude Code, AWS Marketplace and AWS PrivateLink make it easy to drop into existing stacks.
Capabilities
Use Cases
- •Agentic AI backends
Run multi-step agents that chain many LLM calls on fast-decode hardware, which cuts end-to-end task latency and cost per completed task.
- •On-premises enterprise inference
Deploy SambaRack and SambaStack in your own data center, as JPMorganChase plans to, so sensitive data never leaves controlled infrastructure.
- •Sovereign and neocloud token services
Cloud and sovereign-AI operators buy SambaNova infrastructure to resell LLM tokens. SoftBank is the first announced SN50 deployer.
- •Coding assistants and developer tools on open models
Connect Cursor, Cline, Continue or Claude Code to SambaCloud's OpenAI-style API to run open models such as gpt-oss-120b or DeepSeek quickly and cheaply for code generation.
- •Real-time voice agents
Pair SambaCloud with LiveKit, Pipecat, Vapi or ElevenLabs integrations so low LLM latency keeps conversational voice agents responsive.
Ideal For
Best For
- ✓Latency-sensitive agentic workflows that make many sequential LLM calls
- ✓Serving large open-weight models (DeepSeek, MiniMax, Llama, gpt-oss) through an OpenAI-style API
- ✓Regulated enterprises that need on-premises or data-resident inference (SambaStack on-prem / SambaRack)
- ✓Neoclouds and sovereign-AI operators building token-as-a-service businesses on non-GPU hardware
Not Ideal For
- ✗Teams that need frontier proprietary models (GPT, Claude, Gemini Pro) or a broad catalogue: SambaCloud lists only about seven open models, and preview models may be removed at short notice
- ✗Organisations that need model training or large-scale pre-training: SambaNova moved away from training workloads in 2025 to focus on inference
- ✗Buyers who need published compliance attestations (SOC 2, HIPAA) up front: none are listed on the main site or the docs pages reviewed
Deployment
Market Analysis
Pros
- ✓Very fast token throughput on open models. HN users group it with Cerebras and Groq as a top-tier speed provider
- ✓Published per-token pricing and a free tier with no card required, which makes evaluation easy
- ✓Uncommon deployment flexibility: public API, hosted SambaStack, or fully on-premises racks for data residency
- ✓Well funded after the 2026 Series E and F ($1.35B in total) at an $11B valuation, which eases earlier viability concerns
Cons
- ✗Small model catalogue (about 7 models), and preview models 'may be removed at short notice'. HN commenters called it an 'also-ran' on model selection
- ✗Premium pricing on some models: HN users noted DeepSeek R1 at $5.50 when it was cheaper elsewhere, and DeepSeek-V3.x is still $3.00/$4.50 per 1M tokens
- ✗Tight free-tier limits (20 requests/day) and a 20M tokens/day cap on the Developer tier, so production volume requires an enterprise contract
- ✗Corporate turbulence in 2025: layoffs and a training-to-inference pivot, then a reported sale exploration and stalled Intel acquisition talks
Pricing
Free Tier
$0
- ✓No payment method required
- ✓20 requests/min, 20 requests/day, 200K tokens/day
- ✓Access to DeepSeek-V3.1/V3.2, Llama-3.3-70B, gpt-oss-120b, gemma-4-31B-it
Developer Tier
Usage-based (e.g. gpt-oss-120b $0.22 in / $0.59 out per 1M tokens; DeepSeek-V3.1 $3.00 / $4.50)
- ✓Unlocked by linking a payment method
- ✓60-240 RPM per model, 12K-48K requests/day
- ✓20M tokens/day global cap
Enterprise
Contact for pricing
- ✓Custom rate limits via sales
- ✓SambaStack hosted or on-premises
- ✓SambaRack SN40-16 / SN50 hardware
SambaCloud publishes per-million-token prices. Examples from the October 2026 pricing page: gpt-oss-120b $0.22 input / $0.59 output, Llama-3.3-70B $0.60 / $1.20, gemma-4-31B $0.38 / $1.15, MiniMax-M2.7/M3 $0.60 / $2.40 with $0.06 cached input, and DeepSeek-V3.1/V3.2 $3.00 / $4.50. The free tier needs no card but is capped at 20 requests a day. Linking a card unlocks the Developer tier, which has a 20M tokens/day cap. Higher limits, SambaStack and SambaRack hardware are sold through sales, with no list price.
Security & Compliance
Sources
This page was written from 10 sources, 9 on domains other than sambanova.ai.
- 1.sambanova.ai — sambanova.aivendor
- 2.cloud.sambanova.ai — pricing
- 3.docs.sambanova.ai — rate limits.md
- 4.docs.sambanova.ai — sambacloud models
- 5.docs.sambanova.ai — overview.md
- 6.docs.sambanova.ai — introduction.md
- 7.eetimes.com — sambanova abandons intel acquisition raises funding instead
- 8.eetimes.com — sambanova lays off 15 of workforce to refocus on inference
- 9.fnex.com — sambanova strongly considering u s ipo after 1b funding roun
- 10.hn.algolia.com — search
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
Gimlet Cloud
Multi-silicon inference cloud that splits AI agent workloads across GPUs, CPUs and accelerators
DigitalOcean Managed Agents
Managed agent runtime with microVM sandboxes, 16,000+ governed tools and serverless inference, billed on active CPU
Modular
MAX inference framework and Mojo language for serving AI models on NVIDIA, AMD and other chips
ZML/LLMD
Free, Python-free LLM inference server that runs open models on NVIDIA, AMD, Google TPU, Intel and Apple chips from one binary