Union.ai

Union.ai is the commercial runtime for Flyte, the open-source AI and ML workflow orchestrator originally built at Lyft. It gives data and ML platform teams durable, reproducible, crash-resumable pipelines written in plain Python, with a managed control plane while every task execution and every byte of data stays inside the customer's own cloud account.

Union.ai is the commercial runtime built by the creators of Flyte, the Kubernetes-native workflow orchestrator Lyft developed internally from 2016 to compute driver ETAs, open-sourced in 2020 and whose trademark it granted to the Linux Foundation in 2021. Founders Ketan Umare, Haytham Abuelfutuh and George Snelling, who met at Lyft, spun the company out and sell a managed control plane that deliberately keeps execution inside the customer's own cloud. Flyte 2 reached general availability on 4 August 2026 and is the clearest statement of the approach: the DSL layer is gone, workflows are plain async Python where tasks call tasks and fan-out is just asyncio.gather, and an execution trace lets a crashed run resume from the exact step that failed rather than restarting from the beginning. Durability is infrastructure-aware, so a task killed by an out-of-memory error or a preempted spot instance is retried with different resources, not merely a different code path. Union layers a hybrid architecture on top: a Union-managed regional control plane orchestrates work while a data plane operator runs inside the customer's EKS, GKE or AKS cluster, so inputs, outputs, logs and code never transit Union's infrastructure. AWS's own reference architecture credits Union with 10-100x greater task fan-out than open-source Flyte plus spot-instance cost optimisation. The upstream project carries roughly 7,500 GitHub stars under Apache 2.0, and named users include Spotify, Woven by Toyota, Johnson & Johnson and Hopper.

Category

AI Agents & Orchestration

Pricing

Subscription, Usage-based, Contact for pricing

Target Market

CTOs, Data Scientists, Enterprise Developers, MLOps Engineers, Platform Engineers

Deployment

Hybrid, Self-hosted, Open-source, Multi-cloud

Founded

2020

Headquarters

Bellevue, Washington, United States

• ✓
  Pure-Python workflows, no DSL

  Flyte 2 removed the workflow-object DSL entirely; tasks call tasks, loops and conditionals are native Python, and parallel fan-out is asyncio.gather, so pipelines read like ordinary code.

• ✓
  Infrastructure-aware durability and retries

  When a task dies from an OOM kill or a preempted spot node, the runtime changes the resources it requests on retry instead of replaying the same failing configuration.

• ✓
  Execution tracing and crash resumption

  Every action is traced, so a run that crashes mid-pipeline resumes from the exact point of failure — which matters most for non-deterministic AI workloads that are expensive to repeat.

• ✓
  Zero-trust BYOC data plane

  The dataproxy runs inside the customer VPC behind an outbound-initiated tunnel, so inputs, outputs, logs and code never transit the Union control plane or leave the security perimeter.

• ✓
  Content-addressed caching and versioning

  Task outputs are cached and versioned by content, letting teams re-run only the changed portion of an expensive training pipeline and reproduce any past execution exactly.

• ✓
  High-concurrency scheduling

  The platform advertises 50,000-plus actions per run with sub-100ms task startup, and AWS documents 10-100x greater fan-out scale than self-managed open-source Flyte.

• •
  LLM fine-tuning pipelines

  Chain data preparation, distributed fine-tuning and evaluation into one versioned pipeline that resumes from the failed step when a spot GPU node is reclaimed mid-run.

• •
  Bioinformatics and drug-discovery workflows

  Life-sciences teams such as Johnson & Johnson run long genomics and screening pipelines where reproducibility and strict data residency are regulatory requirements, not preferences.

• •
  Production feature engineering

  Recompute and backfill model features on a schedule across large datasets, with content-addressed caching so unchanged upstream steps are skipped rather than re-executed.

• •
  Autonomous-systems and geospatial batch processing

  Customers including Woven by Toyota process large sensor and mapping datasets as massively parallel fan-out jobs coordinated from a single Python entry point.

• •
  Consolidating fragmented orchestration

  Platform teams migrate scattered Airflow DAGs and bespoke Kubernetes jobs onto one typed runtime, gaining shared RBAC, lineage and cost controls across every ML team.

Union publishes real list pricing, which is unusual in this category: the Team plan is $950/month with that amount credited back against usage, then $0.0075 per action, where an action is one end-to-end invocation of a task, trace or condition — retries and status checks are explicitly not billed as extra actions. There is no free managed tier and no advertised free trial; the free path is running open-source Flyte yourself on your own Kubernetes cluster. Enterprise pricing, which is where 3+ clusters, custom concurrency and retention, and zero-trust deployment live, is quote-only with volume discounts, and discounts are offered to startups, universities and non-profits on request. The real cost of ownership also includes the Kubernetes cluster, S3 storage and database that the customer provides and operates.

This page was written from 8 sources, 4 on domains other than union.ai.

1. 1.union.ai — pricingvendor
2. 2.union.ai — securityvendor
3. 3.union.ai — flyte 2 is generally available the durable open source ai ruvendor
4. 4.union.ai — union ai completes 38 1 million series a to power a new era vendor
5. 5.aws.amazon.com — build ai workflows on amazon eks with union ai and flyte
6. 6.techcrunch.com — union ai raises 10m to simplify ai and ml workflow orchestra
7. 7.api.github.com — flyte
8. 8.mlai.qa — prefect vs metaflow vs flyte vs airflow mlops 2026