Baseten
by Baseten
Production inference for open-source, custom and fine-tuned AI models
Baseten is an AI inference platform that deploys open-source, custom and fine-tuned models into production on dedicated GPU infrastructure or pre-optimized token-billed APIs. It targets engineering teams that have outgrown a single model provider and need to control latency, cost and availability across clouds themselves.
Baseten, founded in 2019 and headquartered in San Francisco under CEO Tuhin Srivastava, operates the inference layer that runs after a prompt is submitted, rather than training models of its own. The platform spans four products: Dedicated Inference, which runs custom and fine-tuned models on purpose-built single-tenant infrastructure; pre-optimized Model APIs offering ready-to-serve open models including Kimi K3, DeepSeek-V4-Flash and GLM-5.2 Fast; a training path built on the Loops SDK for frontier reinforcement-learning workloads that promotes directly to production; and Baseten for Model Labs, which gives model developers distribution and monetization infrastructure. Its differentiation is runtime engineering — custom kernels, advanced decoding, TensorRT-LLM optimization, and fast cold starts — packaged as Baseten Embeddings Inference, which the company claims delivers over 2x higher throughput at 10% lower latency, and Baseten Chains for compound multi-model applications. The company follows an asset-light model, brokering capacity across 18 clouds and 87 clusters rather than owning data centers, and reports processing more than one billion inference calls daily. Named customers include Notion, HubSpot, Writer, Harvey, ClickUp, OpenEvidence, Zed Industries, Cursor and Abridge. Revenue reached roughly $600M annualized by March 2026, and in June 2026 Baseten raised a $1.5B Series F across tranches priced at $11B and $13B, co-led by Altimeter Capital, Spark Capital, Sands Capital and Wellington Management, following Nvidia's $150M participation in its January 2026 Series E.
The ML platform or infrastructure engineering team at a product company already serving models in production, which needs predictable latency and multi-cloud GPU availability without building a serving stack in-house.
Production-grade model serving with optimized runtimes and cross-cloud capacity, billed per GPU-minute or per token, without operating your own GPU fleet.
At a Glance
- Category
- Infrastructure & Cloud
- Pricing
- Usage-based, Contact for pricing
- Target Market
- CTOs, Enterprise Developers, Data Scientists, ML Platform Engineers
- Deployment
- Cloud-first, Multi-cloud, Hybrid, Self-hosted, API-based
- Founded
- 2019
- Headquarters
- San Francisco, United States
- Customers
- Not disclosed; named customers include Notion, HubSpot, Writer, Harvey, ClickUp, OpenEvidence, Zed Industries, Cursor and Abridge
Key Features
- ✓Dedicated Inference
Runs custom and fine-tuned models on single-tenant GPU infrastructure with autoscaling and scale-to-zero, so teams control isolation and latency.
- ✓Pre-optimized Model APIs
Token-billed endpoints for open models such as Kimi K3, DeepSeek-V4-Flash and GLM-5.2 Fast, removing any deployment work.
- ✓Baseten Embeddings Inference (BEI)
A tuned embeddings runtime the company claims delivers over 2x higher throughput and 10% lower latency than baseline serving.
- ✓Baseten Chains
An SDK for compound AI applications that composes multiple models behind one endpoint, targeting better GPU utilization and lower end-to-end latency.
- ✓Cross-cloud high availability
Capacity brokered across 87 clusters on 18 cloud providers, so GPU scarcity in one region does not stall production traffic.
- ✓Self-hosted and hybrid deployment
Enterprise customers can run the control plane against their own VPC with optional flex capacity, keeping inference inside their security boundary.
- ✓Training with Loops SDK
Frontier reinforcement-learning training that promotes resulting checkpoints straight into a production deployment on the same platform.
Capabilities
Use Cases
- •Low-latency code completion
Zed Industries serves code-completion models on Baseten and reports roughly 2x faster completions for its editor users.
- •Clinical and medical search
OpenEvidence runs embeddings inference on Baseten, citing roughly 3x speed improvement and about 160ms retrieval latency in production.
- •Real-time meeting transcription
ClickUp uses Baseten for sub-300ms transcription without the unpredictable latency spikes that break live product experiences.
- •Replacing closed-model API spend
Teams route suitable traffic to cheaper open-weight models on dedicated GPUs, cutting per-token cost against frontier vendor pricing.
- •Voice agent pipelines
Streaming speech-to-text and text-to-speech runtimes with low time-to-first-byte support conversational agents that must respond within human turn-taking latency.
Ideal For
Best For
- ✓Serving fine-tuned or custom open-source models in production with low, predictable tail latency
- ✓Migrating high-volume workloads off closed frontier APIs onto cheaper open-weight alternatives
- ✓Real-time speech workloads — transcription, diarization and low time-to-first-byte text-to-speech
- ✓High-throughput embeddings pipelines backing retrieval and semantic search systems
- ✓Compound multi-model applications that chain several models behind one endpoint
Not Ideal For
- ✗Non-technical business teams wanting a finished application — Baseten is infrastructure, and reviewers note it assumes in-house ML engineering rather than delivering a working use case
- ✗Organizations that require fixed, predictable monthly bills; per-minute replica billing means idle replicas and traffic spikes both cost money and make budgeting hard
- ✗Teams whose workload is bursty and idle most of the time, where per-second execution billing from a competitor like Modal is structurally cheaper
- ✗Buyers who only need a single hosted frontier model and no control over the serving runtime
Integrations
Deployment
Market & Ratings
Not disclosed; named customers include Notion, HubSpot, Writer, Harvey, ClickUp, OpenEvidence, Zed Industries, Cursor and Abridge
Market Analysis
Pros
- ✓Measured latency wins in production, with customers citing 2x faster code completions, ~160ms embeddings retrieval and sub-300ms transcription
- ✓Multi-cloud capacity across 18 providers makes GPU availability far less fragile than a single-cloud commitment
- ✓Token-billed Model APIs on open-weight models undercut closed frontier vendors substantially for suitable workloads
- ✓Strong compliance posture for an infrastructure startup — SOC 2 Type 2, SOC 3, HIPAA, GDPR, PCI DSS and CSA STAR Level 1, with a public pen-test report
- ✓Self-hosted and hybrid VPC deployment keeps inference inside the customer's security boundary
Cons
- ✗Per-replica-hour billing on dedicated deployments charges continuously regardless of request volume, so a two-replica redundant setup doubles spend before any traffic arrives
- ✗Independent comparisons put Baseten's effective H100 rate around $6.50/hr against roughly $3.95/hr on Modal, whose per-second execution billing charges nothing while idle
- ✗Reviewers repeatedly flag that this is infrastructure, not a solution — realizing a business use case still requires ML engineers, model work and ongoing MLOps
- ✗Variable per-minute billing makes cost forecasting difficult during traffic spikes, a recurring budgeting complaint
- ✗Analysts name model commoditization and hyperscaler bundling of inference as real medium-term threats to differentiation, alongside GPU supply constraints
- ✗Very little independent practitioner discussion exists — Hacker News threads about Baseten are almost entirely funding announcements with single-digit points and near-zero comments
Pricing
Basic
$0
- ✓Pay-as-you-go GPU and token billing
- ✓Dedicated deployments and Model APIs
- ✓Training
- ✓Fast cold starts
- ✓SOC 2 Type II and HIPAA compliance
- ✓Email and in-app chat support
Pro
Contact for pricing
- ✓Everything in Basic
- ✓Volume discounts on compute
- ✓Priority GPU access
- ✓Dedicated compute and higher rate limits
- ✓Slack and Zoom support with hands-on engineering
Enterprise
Contact for pricing
- ✓Self-hosted and hybrid deployment
- ✓Custom SLAs
- ✓Advanced security controls
- ✓Global regions
- ✓Dedicated forward-deployed engineers
There is no monthly platform fee — Basic is $0 and everything is metered. Dedicated deployments and training bill per GPU-minute: T4 at $0.01052/min, H100 at $0.10833/min and B200 at $0.16633/min, with CPU instances between $0.00058 and $0.01382/min. Model APIs bill per million tokens with separate input, cached-input and output rates; DeepSeek V4 Flash lists at $0.13 input, $0.028 cached input and $0.26 output per million. New accounts receive starter credits. Critically, dedicated deployments bill per replica-hour whether or not requests arrive, so redundancy doubles cost immediately. Volume discounts, self-hosted deployment, custom SLAs and global regions are all Enterprise-gated with unpublished pricing.
Security & Compliance
Sources
This page was written from 8 sources, 6 on domains other than baseten.co.
Stay Ahead of the Curve
Weekly enterprise AI insights for technology leaders. No spam, no vendor pitches—unsubscribe anytime.
SubscribeRelated Products
WEKA NeuralMesh
Microservices storage and memory fabric built for AI training and inference at exabyte scale
Tsuga
Bring-your-own-cloud observability that keeps telemetry, and its cost, inside your own AWS account
OpenObserve
Open-source, Rust-based observability on object storage — logs, metrics, traces and LLM telemetry in one binary
Coralogix
AI-native observability that queries logs, metrics and traces in place — index-free, in your own S3 bucket