For a decade the answer to a spreadsheet was a gradient-boosted tree. Churn, credit risk, claims triage, demand planning — you trained XGBoost or LightGBM on that dataset, tuned it, and shipped it. Every new table got its own model.
That default has now broken, and it broke in a specific, checkable place rather than everywhere at once. The same architectural shift that produced chat models has arrived at structured data, and the enterprise question is no longer whether it works. It is where the boundary sits, because the boundary is what decides whether any of it applies to the tables you actually run.
The crossover has a number on it
The turn is documented in a peer-reviewed venue, which is unusual for this field. TabPFN v2 — a transformer pretrained on roughly 100 million synthetic datasets — was published in Nature 637, 319–326. It does no training on your data at all. It reads your table as context and predicts in a single forward pass.
Its reported margins against the strongest classical baseline are not marginal. In normalized ROC AUC for classification it scored 0.939 against CatBoost's 0.752 in the default setting, and 0.952 against 0.822 when both were tuned. For regression, normalized RMSE ran 0.923 against 0.872 default and 0.968 against 0.875 tuned. It reached those numbers in 2.8 seconds, against ensembles given four hours of tuning.
The original ceiling was 10,000 rows and 500 features — small enough that most production tables fell outside it. That ceiling moved. TabPFN-2.5, released 6 November 2025, handles 50,000 samples and 2,000 features — a 5× and 4× increase — and its authors state it "outperforms tuned tree-based models" while matching AutoGluon 1.4 tuned for four hours.
Amazon shipped the same idea into its own AutoML stack. Mitra is a tabular foundation model pretrained purely on a curated mixture of synthetic priors, available in AutoGluon in three lines with open weights. Amazon reports state-of-the-art results against TabPFNv2, TabICL, CatBoost and RealMLP across TabRepo, TabZilla, AMLB and TabArena.
On TabArena — the living, Elo-rated benchmark built from 51 datasets curated out of 1,053 candidates across 14 prior benchmarks — the top single models are now all pretrained transformers, with the best gradient-boosted-tree configuration roughly 240 Elo below them.
Why trees held the line for a decade
This matters because the resistance was real and well-evidenced, not vendor inertia. The canonical result is Grinsztajn, Oyallon and Varoquaux at NeurIPS 2022, which benchmarked 45 datasets and found tree ensembles beat neural networks at every hyperparameter search budget. The paper named three structural reasons:
- Neural nets are biased toward smooth functions. Real tabular targets are irregular and jagged — a credit threshold is a cliff, not a curve.
- Uninformative features hurt neural nets far more. Business tables are full of columns that carry nothing, and trees ignore them cheaply.
- Rotation invariance is the wrong prior for columns. Mixing "age" and "income" into a linear combination destroys meaning that a tree preserves by construction.
None of those objections were wrong. What changed is that pretraining on tens of millions of synthetic tables lets a transformer learn the irregular, feature-identity-preserving behaviour directly, instead of inheriting the smooth prior that made it lose. The answer was not a better optimizer. It was a different unit of work.
Three caveats decide whether this applies to you
The leaderboard is protocol-sensitive, and TabArena's own authors say so. Their paper states plainly that after post-hoc ensembling neural networks become the strongest single models — but that without it, the top three would all rank below CatBoost. Peak performance "is misrepresented unless post-hoc ensembling is used." A ranking that reorders under a different evaluation protocol is a finding about protocols as much as about architectures. Ask which protocol produced any number a vendor shows you.
The cost profile inverts the comparison. One 2026 cross-model benchmark priced the same workload on a rented B200: roughly $27 of GPU time for the heaviest foundation model, $0.65 for TabPFN-3, and effectively nothing for LightGBM on CPU — plus a 20–25 minute cold-start penalty to compile GPU kernels before the first prediction. Trees still offer sub-second training, microsecond inference, no GPU, no row ceiling and mature tooling. The accuracy frontier moved; the operational case did not.
The ceiling is still a ceiling. Fifty thousand rows is a real constraint. Independent testing on a 230,130-row dataset hit out-of-memory on a 3.8 GB GPU and crashed the kernel on CPU, forcing a 10,000-row sample to run at all. If your table has millions of rows, this is not yet your decision.
Time series shows the same shift, but the win is different
The pattern repeats on forecasting. TimesFM, Google's decoder-only forecasting model, shipped its 3.0 checkpoint on 28 August 2026 under Apache 2.0 and is available inside BigQuery ML and Vertex Model Garden. In zero-shot evaluation it beats statistical baselines like ARIMA and ETS on the Monash archive and matches supervised models trained on the target series, and independent benchmarking across public and proprietary industrial datasets found foundation models outperforming the best deep-learning and statistical baselines.
But the most useful report came from a team that ran these in production rather than on a benchmark. An observability platform evaluating Chronos, TimesFM, Tiny Time-Mixers and Toto against classical baselines on live Kubernetes metrics concluded explicitly that foundation models do not replace classical approaches. The win was operational: one model that generalises across hundreds of streams, instead of hundreds of individually tuned per-stream models. They also found that no model handled genuine regime changes zero-shot — a new traffic pattern or a major configuration change defeated all of them, though the pretrained ones recovered faster.
That is the honest version of this story for an enterprise. The headline is accuracy; the value is that the maintenance burden of a per-dataset model fleet becomes a single deployed model.
What to do this quarter
- Inventory by row count, not by enthusiasm. Split your models into those under ~50,000 rows and those above. Only the first group is in scope today, and for most organisations it is a larger share than expected — risk scorecards, churn models and per-segment forecasts are usually small.
- Run the cheap comparison before the expensive migration. A tabular foundation model needs no training, so the evaluation is a forward pass on data you already have. Anything that takes a week to evaluate is being evaluated wrong.
- Price the inference, not the training. The training saving is real and the inference bill is the offset. Model the GPU-hour cost at your actual prediction volume; a model that is free to fit and expensive to serve can lose to a tree you tune once a quarter.
- Keep the tree as the baseline you must beat. Not as a fallback — as the control. If the foundation model cannot beat a tuned LightGBM on your data under your protocol, the leaderboard is not evidence about your table.
The shift is real, and it is narrower than the headline. The unit of work changed from training a model per dataset to conditioning one pretrained model on each dataset, and that changes hiring, tooling and maintenance more than it changes any single accuracy number. Below the row ceiling, the burden of proof has now moved to the tree.
Continue Reading
- SAP's €1B Bet on the Other Half of Enterprise AI
- Nvidia's $400M Kumo Bet: LLMs Can't Touch Your Database
- vLLM vs TensorRT-LLM vs SGLang: Default to vLLM
- Braintrust vs Langfuse vs Promptfoo: Don't Pay for the Gate
- What RAG Actually Costs: $1,308 a Month at 10M Tokens/Day
- Dreamforce 2026 Shipped Agent Operations, Not the Headline Model