Your GPUs sit idle while teams queue because static quotas strand them, not because you picked the wrong scheduler. Put Kueue on the Kubernetes cluster you already run. It is free, vendor-neutral, and lets one team borrow another's unused quota and return it through preemption. Add MIG slices for notebooks and small inference. Buy NVIDIA Run:ai only if you need GPU fractions with enforced memory limits, or if an NVIDIA AI Enterprise entitlement already covers it, in which case switch it on. Keep Slurm if it already runs your large training jobs. Don't adopt it for a mixed fleet. The loser is the default: the device plugin plus ResourceQuota, which answers over-quota work with a 403, not a queue.
| Option | Verdict | Lends idle quota to other teams? | Fractional GPU | Licence and price (checked 21 Sep 2026) | Don't pick it if |
|---|---|---|---|---|---|
| Kueue | The default for Kubernetes shops | Yes: cohorts, borrowing limits, reclaim by preemption | No; pair it with MIG, or HAMi for memory-enforced slices | Apache 2.0, $0 | You need vendor-supported GPU fractions with enforced memory limits, or a UI for people who don't use Kubernetes |
| NVIDIA Run:ai | Buy it for fractions, or because you already own it | Yes: quotas, over-quota use, fair share | Yes, with memory enforced | Commercial. $2,600/GPU/year listed on AWS Marketplace (private offer only), or bundled in NVIDIA AI Enterprise at $4,500/GPU/year | Any non-NVIDIA accelerator shares the cluster, or all you need is quota borrowing |
| KAI Scheduler | Run:ai's scheduling core, free and without support | Yes: hierarchical queues, DRF fairness, reclaim | Yes, but memory is not enforced unless you add HAMi-core or NvFractions | Apache 2.0, $0 | You need a vendor SLA or a self-service UI |
| Slurm | Keep it for large training; don't adopt it for a mixed fleet | Through fair-share priority and QOS or partition preemption | MIG yes; shards are not fenced | GPL, $0. Support priced on request by SchedMD, now part of NVIDIA | The same GPUs also serve inference services on Kubernetes |
| Device plugin + ResourceQuota | Loses for any multi-team fleet | No. Over-quota pods are rejected with a 403 | Time-slicing (no memory isolation) or static MIG | Apache 2.0, $0 | More than one team competes for the same GPUs |
To compare like with like, every cost and capability below uses one reference fleet: 64 H100 GPUs in eight 8-GPU nodes, shared by five teams. One team runs 16-to-32-GPU gang-scheduled training jobs. Two applied teams run single-node fine-tunes and evals. A data-science group keeps about 30 notebooks open. The platform team hosts a few inference services.
Why Do GPUs Sit Idle While Teams Queue?
GPUs sit idle because a namespace quota works like a wall, not a loan. In Kubernetes you cap a team's GPUs with requests.nvidia.com/gpu in a ResourceQuota, and the documentation is blunt about what happens past the cap: "the control plane rejects that request with HTTP status code 403 Forbidden." There is no queue behind that error, no priority order and no borrowing. If the training team has 16 GPUs of quota and is between runs, the data-science team still cannot use them. Both teams see a shortage while the GPUs sit dark.
Fragmentation is the second cause. It is less obvious, and on a cluster that already lends quota it costs more. Microsoft's study of its Philly training cluster (USENIX ATC 2019: 96,260 jobs over 75 days, October to December 2017) found that for jobs needing more than 8 GPUs, 97.9% of queueing delays were fragmentation delays, not fair-share limits. The GPUs existed, but they were scattered across servers. The authors also found that when two-thirds of the cluster's GPUs were in use, fewer than 4.5% of servers were completely empty. A job that needs a whole 8-GPU node waits behind a crowd of single-GPU jobs, each taking one slot on a different machine.
The third cause is that "allocated" does not mean "used". Philly already lent idle quota between groups and preempted jobs when the cluster reached 90% occupancy. Even so, the same paper measured the hardware utilisation of GPUs actually in use at about 52% on average. A scheduler controls who gets a GPU. It does not control how well the job uses the GPU once it has it.
How Much Utilisation Does a Better Scheduler Actually Buy?
Measured against a static-allocation baseline, the gain is large. On a cluster already running large gang jobs, it is close to nothing. The gain depends entirely on your baseline, so read every figure with its baseline in mind.
The best-known figure is Run:ai's own. Its case study describes an unnamed facial-recognition company with 24 NVIDIA DGX servers and 30 researchers on two continents. Before Run:ai, "GPU resources were statically allocated", and utilisation was 28%. Afterwards it was "over 70%". The charts cover five to six weeks on each side. Treat it as a vendor claim: the customer is anonymous, the metric is never defined, and the headline says 73% while the results panel says 70%. Its baseline is still the useful part. The gain came from moving teams off static, per-team allocation onto a shared pool. Any scheduler that lends quota, Kueue included, attacks the same baseline.
Now the counter-evidence. Shanghai AI Laboratory published six months of traces (March to August 2023) from two A100 clusters (NSDI 2024 paper). Seren has 2,288 GPUs and Kalos has 2,416. The lab's own scheduler, with quota reservation and preemptible best-effort jobs, runs on Slurm on the first and on Kubernetes on the second. Median GPU utilisation was 97% on the Slurm cluster and 99% on the Kubernetes one, against 48% for Philly and 4% for Alibaba's PAI. The underlying scheduler made no visible difference. The authors attribute the high numbers to the workload, transformer-based LLMs with similar architectures, which suggests large LLM training keeps GPUs busy whatever schedules it.
Define the metric before you buy anything on its strength. GPU utilisation, as nvidia-smi reports it, is the percentage of time in which at least one kernel was executing. It does not measure how much of the chip was working. The same Shanghai paper found median SM activity of about 40% on clusters that reported 97-99% "utilisation". A scheduler can raise the first number. Only the job's own code can raise the second.
The upshot for your purchase: a scheduler's real leverage is the small, bursty and interactive part of your fleet — notebooks, evals and single-GPU fine-tunes. If that slice is small, no scheduler will move your numbers much.
What Each Option Costs on a 64-GPU Fleet
Of the five options, only Run:ai carries a licence fee, and it pays for itself only if it recovers about 7.4% of the fleet's useful capacity. To price the reference fleet, we use Lambda's on-demand H100 rate of $3.99 per GPU-hour (checked 21 September 2026) as a stand-in for GPU cost. Running 64 GPUs around the clock at that rate comes to about $2.24 million a year.
| Option, 64 GPUs | Licence per year | Per GPU-hour | Share of the $2.24M fleet cost |
|---|---|---|---|
| Device plugin + ResourceQuota | $0 | $0 | 0% |
| Kueue | $0 | $0 | 0% |
| KAI Scheduler | $0 | $0 | 0% |
| Slurm | $0 for the licence; support priced on request | n/a | 0% plus support |
| Run:ai, AWS Marketplace list price | $166,400 | $0.30 | 7.4% |
| NVIDIA AI Enterprise, which includes Run:ai | $288,000 | $0.51 | 12.9% |
| NVIDIA AI Enterprise, billed hourly in the cloud | $560,640 | $1.00 | 25.1% |
Sources: NVIDIA Run:ai on AWS Marketplace ($2,600 per GPU per year, "only available via private offer"). NVIDIA AI Enterprise pricing ($4,500 per GPU for one year, $18,000 for five, $22,500 perpetual with five years of support, or $1 per GPU-hour plus the cloud instance). That page was last updated 2 September 2026. SchedMD's support page publishes no prices and asks you to email sales.
The break-even is the number to take into the room. At $3.99 an hour, Run:ai's list price pays for itself if it lifts useful work by 7.4% over whatever you would otherwise run. Compared with doing nothing, that bar is easy: a 28% baseline only has to reach about 30%. Compared with Kueue, which removes the static-quota baseline at no cost, Run:ai has to earn the 7.4% from what Kueue cannot do. In practice that means fractional GPUs for the notebook slice. If 16 of your 64 GPUs sit under notebooks, Run:ai must recover about 30% of that slice to pay back.
Owned hardware makes the maths harder, not easier. Your amortised cost per GPU-hour is almost certainly below a neocloud's on-demand rate, so a fixed per-GPU licence takes a larger share of it. For how the rate you are quoted differs from what a GPU-hour actually costs, see our GPU-hour pricing teardown.
Kubernetes Device Plugin and ResourceQuota: When Plain Quotas Are Enough
Plain quotas are enough when your teams don't compete for the same GPUs, and you can check that with a week of data. Pull kube_resourcequota (hard limit against used) per namespace from kube-state-metrics for seven days. If no namespace ever hits its hard cap while another sits below half of its own, a scheduler has no stranded capacity to recover. The usual cases are a single team, teams that each own a dedicated node pool, and inference services with steady, predictable replica counts. In those setups, installing a queueing layer adds work and returns nothing.
The NVIDIA device plugin (Apache 2.0) advertises whole GPUs as nvidia.com/gpu and tracks their health. A pod cannot request half a GPU. To share, you choose between two mechanisms. Time-slicing advertises one GPU as several replicas. The GPU Operator documentation warns that "there is no memory or fault-isolation between replicas", and a pod that requests two replicas is not guaranteed proportionally more compute than a pod that requests one. MIG exposes hardware slices with names like nvidia.com/mig-1g.5gb, using either the single or mixed strategy.
Time-slicing also has an operational trap. Karpenter could not scale out for time-sliced pods, because it counted physical GPUs while the pods requested virtual replicas. According to a February 2026 comment on the issue, the fix is an alpha NodeOverlay resource in Karpenter v1.8 and later, which you have to declare by hand.
The device plugin's replacement is arriving, but not all of it is ready. Dynamic Resource Allocation graduated to GA in Kubernetes 1.34. In Kubernetes 1.37, DRA drivers can now satisfy classic extended-resource requests, the nvidia.com/gpu kind, without a separate device plugin. The capability that matters most here is creating MIG slices on demand, and it is still immature: NVIDIA's DRA driver ships dynamic MIG as alpha, off by default.
Don't rely on this option if more than one team competes for the same pool. ResourceQuota is a ceiling, not a scheduler. That is why the default setup loses this comparison: it produces exactly the baseline, static and never shared, that every vendor case study starts from.
Kueue Is the Default for Kubernetes Shops
Kueue fixes the problem that actually strands GPUs, which is quota nobody can lend, and it does so without a licence fee or a new scheduler. In the project's own words, Kueue is "a kubernetes-native system that manages quotas and how jobs consume them". It is an Apache 2.0 kubernetes-sigs project. It decides when a job is admitted, and it leaves the choice of node to the standard kube-scheduler.
The mechanics map directly onto the idle-while-queued problem. ClusterQueues grouped into a cohort "can borrow unused quota from each other". A borrowingLimit caps how much a queue can take, and a lendingLimit caps how much of its idle quota a queue will give away. reclaimWithinCohort lets the owner preempt a borrower to get its GPUs back. Admission is all-or-nothing, so a 32-GPU job never sits holding 31 GPUs. Topology-aware scheduling and multi-cluster dispatch (MultiKueue) cover the placement and scale-out cases. Quota is declared per resource name, so full GPUs and MIG profiles can carry separate quotas in the same queue.
It runs in production at scale. Netflix reported in June 2026 that Kueue has largely replaced the custom queuing and scheduling logic in its in-house batch system, and it named preemption, all-or-nothing scheduling and topology-aware scheduling as the features it wanted. CoreWeave now offers Kueue on its managed Kubernetes, describing borrowing as automatic, "with resources returning to their original owners when needed."
Here is what breaks. An open issue, kueue#10815, describes fair-sharing preemption that ignores topology. Kueue picks borrowers to evict without regard to topology, so the GPUs it would free are "scattered across different cliques". Topology validation rejects that plan, and the topology-constrained job stays pending anyway. If your large jobs need a whole NVLink domain, test that exact case before you rely on reclaim.
Don't pick Kueue on its own if your data scientists need GPU fractions with enforced memory limits. Kueue handles admission and does not partition GPUs. Pair it with MIG; with HAMi, an open-source CNCF Sandbox project whose memory-enforced GPU slices Kueue can queue against quota, but which you run and support yourself; or look at Run:ai. Also skip it if your users won't touch YAML. Kueue is driven through Kubernetes objects, with no researcher-facing job portal, and "ask the platform team" does not scale to 30 researchers.
NVIDIA Run:ai Earns Its Price on Fractional GPUs, Not Quotas
Run:ai is the only option here that gives notebooks fractional GPUs with enforced memory limits, plus a UI that non-Kubernetes users will actually use. Those two things are what you are paying for. Everything else it does, including quotas, over-quota use, fair share and gang scheduling, you can now get free from Kueue or from Run:ai's own open-sourced scheduler.
Fractions are the real differentiator. Run:ai's documentation says "each pod uses its own separate virtual memory address space" and that "no workload can use more than requested". Time-slicing gives neither guarantee. Dynamic fractions go further: a notebook can request 0.25 of a GPU with a limit of 0.80 and borrow memory up to that limit while nobody else needs it. For 30 notebooks that mostly sit idle, this is the feature that pays the licence.
Read the limits before you sign. By default, fractions share compute through NVIDIA's time-slicing. Fractions and MIG cannot run on the same node. Dynamic MIG was deprecated starting in v2.19, so MIG layouts are now fixed and "changing MIG profiles requires administrative intervention." The system requirements list only NVIDIA GPUs, do not support vGPU, and cover Kubernetes 1.34 to 1.36. Upstream has already shipped 1.37, so Run:ai will be one of the things that sets your Kubernetes upgrade schedule.
Don't pick Run:ai if any non-NVIDIA accelerator is on your roadmap for this cluster. It supports NVIDIA GPUs only, and the case for silicon diversification is getting stronger. Also skip it if your fleet is mostly large gang-scheduled training, which already runs at 97-99% on any scheduler, or if all you need is quota borrowing. That last purchase is the most expensive mistake available on this page: $166,400 a year at list price for a feature Kueue gives you free.
Do pick it without further deliberation if you already own it. NVIDIA AI Enterprise now includes Run:ai, both self-hosted and SaaS, and NVIDIA said in February 2025 that every system sold with the H200 NVL comes with a five-year AI Enterprise subscription. Check your purchase orders before you run any bake-off.
KAI Scheduler Gives You Run:ai's Scheduler Without the Licence
KAI Scheduler is the scheduling engine taken out of Run:ai, released as open source. Choose it if you want Run:ai's queue semantics and can live without the control plane and the support contract. NVIDIA open-sourced it on 1 April 2025 under Apache 2.0, and the blog post says it "continues to be packaged and delivered as part of the NVIDIA Run:ai platform." It is now a CNCF Sandbox project with hierarchical queues, Dominant Resource Fairness, reclaim and preemption, gang scheduling, bin-packing, topology-aware placement, DRA support and time-based fair share.
Unlike Kueue, KAI replaces the scheduler rather than sitting in front of it. That gives it more control over bin-packing, which is the fragmentation problem Philly measured. It also means you are running a second scheduler in production.
Its GPU sharing is where it differs most from the paid product. KAI's own GPU-sharing documentation says that in its non-enforced mode, containers "may see and use more GPU memory than requested". To enforce memory limits you must add HAMi-core or NVIDIA's NvFractions runtime path. So the free version of fractional GPUs comes with a memory-isolation project you build yourself.
Don't pick KAI if you need someone to call when the scheduler misbehaves at 3am, or if your users expect a portal. Choose it when your platform team is strong and wants bin-packing that Kueue on top of kube-scheduler won't give you.
Slurm Still Wins Large Training, and Loses the Mixed Fleet
If your fleet is mostly multi-node training and your users already write sbatch scripts, Slurm is the right answer and you should not migrate for fashion. Slurm is GPL-licensed and, per NVIDIA, runs more than half of the top 10 and top 100 systems on the TOP500 list. Gang allocation, backfill, topology-aware placement and multifactor fair-share priority are native to it. They are not add-ons. Preemption can be driven by partition priority tier or by QOS, and a preempted job can be cancelled, requeued or suspended.
Practitioners summarise the trade-off accurately. SkyPilot's comparison (July 2025) puts the gang argument this way: "When you're training a 70B parameter model across 64 GPUs, you can't start with 63 and hope the last one shows up eventually." The same post quotes a Nebius engineer on the cost: "all nodes must be identical (Linux user and group IDs, software versions and the like)." SkyPilot builds a layer that runs on top of both, so read its framing with that in mind; we list it here.
Slurm is weaker on GPU sharing. It has supported MIG since version 21.08. But its generic sharing mechanism, shards, "does not fence the processes running on the GPU", and NVIDIA MPS allows only one user's MPS server per node at a time. For 30 notebooks, Slurm gives you whole GPUs or MIG slices and nothing in between.
If you need both, you no longer have to run two platforms. NVIDIA's Slinky slurm-operator runs full Slurm clusters as pods on Kubernetes. NVIDIA runs it in production on clusters with more than 8,000 GPUs and reports NCCL benchmarks matching bare-metal Slurm. The catch, per the same post, is that it "currently assumes one worker pod per node", which over-provisions when your jobs are single-node.
Don't pick Slurm if the same GPUs also serve inference, or if your platform team is Kubernetes-native and has nobody who has run an HPC scheduler. Adopting Slurm for a new mixed fleet means building a second operating model to save a few YAML files.
Fractional GPU vs MIG: What Each Option Actually Isolates
Only MIG isolates memory and faults in hardware. Run:ai fractions, and KAI with HAMi-core, enforce memory limits in software. Time-slicing and Slurm shards enforce nothing. MIG splits a GPU into "up to seven separate GPU Instances", each with its own path through the memory system and its own fault isolation. The cost is rigidity: the slice sizes are fixed in advance.
| Sharing mode | Memory isolated? | Fault isolated? | Resize without an admin? | Where you get it |
|---|---|---|---|---|
| MIG | Yes, in hardware | Yes | No (dynamic MIG is alpha in DRA, deprecated in Run:ai) | Device plugin, DRA, Slurm, Run:ai, Kueue (as quota) |
| Run:ai fractions | Yes, in software | Not documented | Yes, any size | Run:ai only; not on MIG nodes |
| KAI GPU sharing | Only with HAMi-core or NvFractions | Not documented | Yes | KAI Scheduler |
| Time-slicing | No | No | Replica count set in config | Device plugin / GPU Operator |
| Slurm shards | No: "does not fence" | No | Set in config | Slurm |
The practical rule: use MIG for inference and for anything shared between teams, and use software fractions only within one team's notebooks, where the worst case of a memory overrun is a colleague's restarted kernel.
What Nvidia Owning Run:ai and SchedMD Changed for Licensing
NVIDIA now owns the companies behind two of the three most common answers to this question. Neither licence has got worse, but you should price in the loss of vendor neutrality.
Run:ai. NVIDIA completed the roughly $700 million acquisition in December 2024, after EU review. The founders said they would open-source the software to "extend its availability to the entire AI ecosystem", and noted that it then supported only NVIDIA GPUs. What NVIDIA actually open-sourced was the scheduler (KAI, April 2025). The control plane, the UI and the memory-enforced fractions are still commercial, and the product is now included in NVIDIA AI Enterprise. For many buyers the practical effect is a lower price: if you bought H200 NVL systems, you have probably already paid for Run:ai. The cost is a tie to NVIDIA: today's documentation still supports NVIDIA GPUs only.
Slurm. NVIDIA announced its acquisition of SchedMD on 15 December 2025 and committed to "continue to develop and distribute Slurm as open-source, vendor-neutral software." The GPL makes that credible: the licence cannot be withdrawn from code already released. The open question is who sets the roadmap. SchedMD's support page now opens with "SchedMD is now a part of NVIDIA."
The fully neutral option is Kueue, a kubernetes-sigs project that no chip vendor controls. We have seen this pattern before: Ray stays open under the PyTorch Foundation while the hosted control plane above it changes owner as Nscale buys Anyscale, and Nvidia's signed Hugging Face deal would move the Optimum adapters for Trainium, Gaudi and ROCm inside Nvidia. The open-source layer is what survives a change of owner. Put your queue policy there.
How to Choose: The Criteria That Predict Regret
Five questions decide the answer, and the first one settles it more often than the other four combined.
- Does one team hit its quota while another's sits idle? If not, plain quotas are enough and you should stop here. If so, the cause is static quota, and Kueue fixes it for $0.
- What share of your GPUs sits under notebooks and small jobs? Your large training jobs already run close to 100% on any scheduler. The small, bursty slice is the only place fractions pay. If that slice is under a few GPUs, MIG covers it.
- Do you already hold an NVIDIA AI Enterprise entitlement? If yes, Run:ai costs you nothing extra. Turn it on and use it for the notebook slice.
- Will a non-NVIDIA accelerator join this cluster within the hardware's life? If yes, rule out Run:ai as your control plane. Use Kueue, which schedules whatever resources your nodes advertise.
- Can your preemptible jobs resume from a checkpoint? Borrowing only works if the lender can take its GPUs back. Philly preempted jobs by checkpointing the model. A borrowed job that cannot resume loses all its progress when it is preempted, and that teaches teams to hoard quota again.
What changes the answer: NVIDIA's DRA driver taking dynamic MIG out of alpha would narrow Run:ai's lead on fractions. A Run:ai price rise, or new limits on non-NVIDIA hardware, would widen Kueue's.
What to Do in the Next 90 Days
This Week:
- Export seven days of
kube_resourcequota(hard limit and used) per namespace, next to DCGM's SM-activity metric per namespace. Record, in writing, how many GPU-hours sat idle while another team had pending pods. - Ask procurement whether any GPU purchase came with an NVIDIA AI Enterprise subscription. If one did, you own Run:ai.
- Count the GPUs currently assigned to notebooks and dev pods. That number bounds everything Run:ai can earn.
This Month:
- Install Kueue. Put the two teams that compete most into one cohort, with nominal quota equal to their current ResourceQuota, a
borrowingLimit, andreclaimWithinCohortenabled. - Make a working checkpoint the condition for borrowing: a job that cannot resume runs only within nominal quota.
- Split one node pool into MIG slices for notebooks and small inference, and give those slices their own Kueue quota.
Before Q4 Close:
- Re-run the week-one measurement. If borrowed GPU-hours are substantial and queue waits have fallen, stop. If notebook GPUs still show low SM activity, price Run:ai against that slice using the 7.4% break-even, not against the whole fleet.
- Write down your exit plan: which workloads would break if Run:ai or Slurm terms changed, and how long moving them to Kueue would take.
The Bottom Line
This is the server-virtualisation argument again. Data-centre servers used to run mostly idle because every application owned its own box, not because the hypervisor hadn't been invented. Pooling fixed it, and the hypervisor was simply the tool that made pooling possible. GPUs are at the same point. The measured gains in this category come from ending static ownership, and the cheapest tool for that is free. Pay for more only when a specific workload, usually 30 notebooks on 80 GB cards, needs isolation that MIG is too rigid to give.
A scheduler cannot create GPUs. It can only stop a quota from hiding them.
Continue Reading
- GPU Hour Pricing: A $3.44 Rate Cost $68.80 to Use
- 5% GPU Utilization: The $401B AI Capital Bonfire
- GPU Clouds Compared: Nebius on Price, CoreWeave at 3AM
- vLLM vs TensorRT-LLM vs SGLang: Default to vLLM
- NVIDIA Alternatives for Inference: Only Trainium Pays Off
- Ray Is Open Source. The Control Plane Above It Isn't.
- Red Hat Bets Kubernetes Wins Enterprise AI's Control Plane