Pick Azure for On-Demand Confidential H100 Inference at $6.98/Hour

If you need attested confidential GPU inference this quarter, default to Azure's NCCads H100 v5, because it is the only one of the three clouds that sells an H100 with GPU confidential mode on demand. Google's confidential A3 works but can only be bought as Spot or flex-start capacity. AWS has no equivalent: a Nitro Enclave gets CPUs and memory, and the GPU stays outside it. Most teams asking this question don't need any of them, and a zero-retention managed endpoint plus redaction covers their real threat at a fraction of the work.

Workload for every comparison below: one H100-class GPU, Linux, 730 hours a month, serving one model of Llama 3.1 8B to 70B scale. Prices were checked on November 2, 2026.

	Azure NCCads H100 v5	Google Confidential VM A3 High	AWS Nitro Enclaves
GPU inside the TEE	Yes, 1 H100 NVL, 94 GB	Yes, 1 H100, 80 GB	No, CPU and memory only
CPU TEE	AMD SEV-SNP	Intel TDX	Nitro Hypervisor isolation
How you buy it	On-demand or Spot	Spot or flex-start only, no reservations	Any supported parent instance
Price per hour	$6.98 on-demand, about $1.29 Spot (East US 2)	About $6.59 Spot for a standard VM (us-central1); confidential and flex-start rates not confirmed	No enclave fee; p5.4xlarge is $6.88, but its GPU is unprotected
Monthly at 730 hours	$5,095	About $4,809 at the standard-VM Spot rate, if it stays up; confidential rate not confirmed	$5,022 for a GPU outside the boundary
Verdict	Default pick	Pick if you are on GKE or need Confidential Space	Loser for GPU inference

---

What Is Generally Available on Each Cloud?

Azure and Google both sell an attested H100, and AWS does not. Confidential inference here means running a model inside a trusted execution environment (TEE), a hardware-isolated region whose memory the host cannot read, with the boundary extended from the CPU to the GPU.

Azure's NCCads H100 v5 series has one size: 40 vCPUs, 320 GiB of RAM and one 94 GB H100 NVL, with the TEE spanning the confidential VM and the GPU. Microsoft's onboarding repository lists East US 2, Central US and West Europe, though the Retail Prices API also prices the size in South Central US, so run az vm list-skus to see what your subscription can reach. Resizing isn't supported, Ubuntu 22.04 is the qualified image Microsoft names, and high availability is explicitly your job. One GPU per VM is the ceiling. The series page also marks live migration and memory-preserving updates as not supported, and Microsoft says quotas aren't capacity guarantees, so on-demand pricing doesn't promise you a GPU when you ask for one.

Google announced general availability of Confidential VM and Confidential GKE Nodes with H100s on August 29, 2025, on a3-highgpu-1g in three zones: europe-west4-c, us-central1-a and us-east5-a. The catch is in the setup docs: A3 High supports only Spot and flex-start provisioning. The supported configurations page adds no reservations, no live migration and no multi-node clusters. That is a hard problem for an always-on endpoint with an SLA.

Google's other option is g4-standard-48 with an RTX PRO 6000, which does allow on-demand and reservations. Google's docs list it on AMD SEV, where Azure uses SEV-SNP. It also incurs a licence fee for NVIDIA's confidential software, which Google prices on its separate Confidential VM pricing page. HGX B200 and GB300 access is limited to "specific eligible customers" through your account team.

Confidential Space, Google's locked-down runtime for multi-party data, is a separate decision. Google's August 2025 post put it with H100s in preview. The current overview lists TDX with NVIDIA Confidential Computing as a hardware option without stating its status, so get that in writing before you plan around it.

On AWS, a Nitro Enclave is a VM made by partitioning memory and vCPUs from a parent instance, with vsock as its only channel out. The trap is that the requirements page lists P5, P5e, P5en and P6-B300 as supported parents, which reads as GPU support. It means the parent can host an enclave. Nothing in the docs gives that enclave a GPU, and AWS's own LLM-in-an-enclave walkthrough ran Bloom 560m on 8 CPUs of an r5.8xlarge. We found no AWS documentation of an NVIDIA Confidential Computing instance as of November 2, 2026.

The strongest case for AWS deserves stating. AWS says there is no mechanism for any operator to access a Nitro instance or data sent to an ML accelerator or GPU. If your concern is AWS staff and you accept that as a design assurance, an ordinary P5 may satisfy you. What you don't get is evidence about the GPU that a third party can verify, and that page describes attestation only for the instance, through NitroTPM and Attestable AMIs.

---

How Remote Attestation Works, and Who Verifies It

Remote attestation is signed hardware evidence of what is running, checked against reference values before anyone releases a secret to the machine. On a confidential GPU, two separate pieces of evidence have to pass: one from the CPU TEE and one from the GPU.

The CPU report goes to a verifier. On Azure, Azure Attestation validates the SNP report at boot and issues the token that releases keys from Key Vault or Managed HSM. Microsoft says critical operations run in SEV-SNP containers to keep itself out of the trusted base, and documents how a relying party can check that. On Google, you write your own verifier or use Google Cloud Attestation, and you can compare the service's token against your own policy.

The GPU report goes somewhere else. NVIDIA's cloud service, NRAS, validates GPU evidence against firmware reference values, or you run a local verifier. Azure's repository says local attestation is moving to NVIDIA's nvattest, which it labels preview with no SLA. Google documents a live failure mode: attestation can fail on a mismatched record at index 9, and only a full VM stop and restart clears it. Your release policy needs to handle that case, or a restart becomes an outage.

So the honest answer to "who verifies" is whichever party you decide to trust, and that party also needs current reference values. Ryan Hurst's April 2026 review says vendors rarely publish signed measurements per release, so organisations end up building their own reference databases and golden image registries. He also cites an IDC survey, which he describes as ranking attestation validation the top adoption challenge. The release on that study, which the Confidential Computing Consortium commissioned from more than 600 IT leaders, lists it at 84% among several challenges without calling it the top one. Budget engineering time for it.

Attestation also proves less than buyers assume. NVIDIA's threat model says it doesn't show that approved code is free of vulnerabilities, and the overview says it supports a trust decision without replacing the policy behind one.

---

What Does NVIDIA Confidential Mode Cost in Performance?

Published tests put the throughput penalty anywhere from 4-8% to about 20% with one loaded model, and much worse if you swap models. Four published measurements, one of them from a vendor, set that range.

• Phala Network's benchmark, using vLLM on an H100 NVL with SEV-SNP, measured a 6.85% tokens-per-second loss on Llama 3.1 8B, 4.58% on Phi-3 14B and effectively zero on a 4-bit Llama 3.1 70B. Time to first token rose 19% on the 8B model. Phala sells confidential compute, so read this as a vendor-authored paper, though the raw tables are published.
• ETH Zurich's study measured 4-8% on H100, shrinking as batch and input sizes grow, and argues CPU TEEs can be more cost-effective or more secure than GPU TEEs in some cases.
• A May 2026 preprint by Mozilla authors ran Mistral-7B and Qwen3-30B-A3B on Google's a3-highgpu-1g under TDX, the VM this guide covers. At a fixed request rate, non-confidential mode had 17.7% and 21.1% higher global token throughput, and average time to first token was 21.8% and 27.8% higher in confidential mode. In closed-loop tests the gap was 11.5-20.2%, and the larger model saturated earlier. The authors suggest reserving 15-25% extra capacity as a starting point, and they warn against generalising from two models with Secure Boot off.
• A study of relaxed batch inference with model swapping on one H100 found non-confidential mode had 45-70% higher throughput and 20-30% lower latency. The authors blame encryption during model loading.

Phala attributes its overhead to CPU-GPU I/O, and the swap study points to encrypting data on its way to the GPU, which suggests the cost tracks how much crosses that boundary. Phala's and ETH's numbers fall as models and inputs grow, but the Mozilla run found the larger model worse on latency and did no profiling, so don't assume a big model escapes the penalty. The studies used different CPUs, TEEs and models, so the spread doesn't tell you Azure's SEV-SNP is faster than Google's TDX. A multi-model endpoint that swaps weights on demand is the worst case, so pin one model per VM. NVIDIA also lists MIG, MPS and GPUDirect RDMA as unsupported in these configurations, which removes the usual ways of sharing a GPU.

Memory sets a harder limit than overhead. A 120B model at 4-bit is about 60 GB of weights by our arithmetic (0.5 bytes per parameter), which fits a 94 GB H100 NVL or an 80 GB H100 with limited room for KV cache. A 1T model at 4-bit is about 500 GB and fits neither. Multi-GPU is where the threat model gets worse: NVIDIA says Hopper's multi-GPU mode is unencrypted over NVLink, while Blackwell's encrypts it, and Blackwell is the tier Google restricts to eligible customers. Check what you plan to run against our self-hosted break-even numbers before you commit to the architecture.

---

What Is the Price Premium for Confidential Instances?

On Azure on-demand there is no list premium at all. As of November 2, 2026, the Azure Retail Prices API lists Standard_NCC40ads_H100_v5 on Linux at $6.98 per hour in East US 2, $7.89 in Central US and $8.90 in West Europe. The non-confidential NC40ads is also $6.98 in East US 2.

You pay in other ways. At that same price, the confidential size has an 800 GiB temp disk and no Accelerated Networking, while the standard size has 3,576 GiB and supports it. Throughput is the other cost: a 6.85% tokens-per-second loss raises cost per token by about 7.4%, and the 17.7-21.1% gap measured on Google's A3 would raise it by the same 17.7-21.1%.

The Spot figures run the other way, and you shouldn't build on them. The same API shows confidential Spot at about $1.29 per hour in East US 2 against $3.54 for standard. Spot prices move, and an evicted confidential VM has to reattest and reload encrypted weights, which the swap study above identifies as the slow part.

Google is the gap in this comparison. Its accelerator pricing page and its Confidential VM pricing page, which has a section on confidential GPUs, did not return usable rates when we fetched them, so we could not confirm the confidential charge on A3 or a flex-start rate. A third-party tracker observed on November 2 lists a3-highgpu-1g in us-central1 at $11.06 on-demand and $6.59 Spot, and the on-demand rate isn't sold in confidential mode at all. Treat those as standard-VM reference points that may exclude confidential charges, and get your confidential rate from the pricing calculator or your account team.

AWS charges nothing extra for enclaves. A p5.4xlarge with one 80 GiB H100 is $6.88 on-demand per Vantage's tracker, or $5.97 per hour through Capacity Blocks per AWS, which is useful only as a comparison because none of it buys a protected GPU.

---

What the TEE Does Not Protect

The boundary covers the VM and the GPU, and most of the places your sensitive data leaks sit outside it or are trusted by it. NVIDIA's own threat model is direct: guest administrators and software inside the VM remain trusted, the boundary doesn't isolate a user's data from the tenant, and network and storage protection is separate from GPU attestation. A plain confidential VM is designed to keep the cloud operator out, and your own root user stays inside the trust boundary.

Walk the request path and mark where plaintext appears:

1. Your gateway. If LiteLLM or another gateway terminates TLS outside the VM, prompts are plaintext there, and so are its caches and traces.
2. Retrieval. A Qdrant or Pinecone index outside the boundary holds your documents as chunks and embeddings, and retrieved context is plaintext wherever it is assembled.
3. Serving logs. vLLM's request logging is off by default, but with --enable-log-requests and DEBUG level it writes prompt inputs, and --max-log-len defaults to unlimited. Ship those logs outside the TEE and the prompts leave with them.
4. Telemetry. OpenTelemetry's GenAI content capture is opt-in and flagged as sensitive, per TrueFoundry's summary, so check who enabled them. Tools like Langfuse store whatever you send.
5. The serving software itself. A remote code execution bug in your inference server runs inside the boundary with full access, which is why the October vLLM advisories still matter here.

Then there are the attacks the vendors exclude. TDXRay, an IEEE S&P 2026 paper, assumes a malicious cloud operator controlling the host and reconstructs prompts from cache behaviour during tokenization. Its authors note that this runs on the CPU before inference, so "GPU confidentiality offers no protection." They tested Sapphire Rapids, Emerald Rapids and Granite Rapids Xeons, and Google lists Sapphire Rapids for A3 High. AMD's bulletin on the same work treats these techniques as known and out of scope for SEV-SNP, so don't read Azure's AMD platform as immune.

TEE.fail needs physical access and a DDR5 interposer built for under $1,000, and it extracts attestation keys on Intel and private signing keys from an SEV-SNP VM on AMD. Because NVIDIA's reports depend on the CPU VM for trust and aren't bound to a specific one, a demo the researchers cite relayed attestation from a GPU the attacker didn't own. Intel and AMD both class interposers as out of scope. If your adversary is a well-resourced insider at the cloud provider, a TEE reduces that risk without removing it.

---

Who Does Not Need Confidential GPUs at All?

If nobody outside your company is demanding hardware evidence, and your threat is data retention or vendor training, skip this and use the cheaper controls. Many regulated buyers are in this group, because the questions they face concern retention, access and residency, which a TEE doesn't settle.

Threat you face	Control that covers it	Why a TEE adds little
Vendor retains or trains on prompts	Zero-retention mode, written into the contract	The TEE doesn't change what your own stack stores
Model provider sees your data	Amazon Bedrock deployment accounts	Providers have no access to prompts
PII reaching a model	Redaction at the gateway	Works for any model on any cloud
Your own staff reading prompts	IAM, log controls, no DEBUG in production	A plain confidential VM still trusts guest admins
Data leaving a jurisdiction	Regional deployment and residency terms	Attestation says nothing about lawful access

Bedrock's docs state model providers have no access to customer prompts and completions, and its data_retention_mode: none can be enforced across an organisation with a Service Control Policy. Read the model list first: Claude Fable 5 and 5.1 require aws_review, which means up to 30 days of retention with possible AWS human review, unless your account is approved for zero retention. Redaction costs about $40 for 10 million chats with Google Sensitive Data Protection. Our inference provider comparison covers zero retention on open-weight endpoints.

The cases that do justify it are narrower. A hospital, bank or data partner contractually requires proof that you can't read their records. Two parties want to run a model on pooled data neither may see, which is the Confidential Space case. You are protecting proprietary weights on infrastructure you don't control. Or your written threat model names the cloud operator and your regulator accepts attestation evidence for that.

---

Who Should Not Pick Each Option

Skip Azure NCCads H100 v5 if you need more than one GPU per VM, since the series has one size, or if your data must stay in a region the series doesn't reach. Skip it if your platform runs on another cloud and the cross-cloud egress and identity work would exceed the value of on-demand capacity. Teams that want managed Kubernetes-native rollout should compare Google's GKE support first.

Don't choose Google Confidential VM A3 High for an always-on production endpoint with an availability SLA, because Spot can be preempted and reservations aren't available. It suits batch jobs against sensitive data, and it is the better pick if you already run GKE, since Confidential GKE Nodes with H100s are generally available on Standard and Autopilot. It also has Confidential Space as a purpose-built multi-party runtime, subject to that status check.

Don't pick AWS Nitro Enclaves for GPU inference, because the GPU isn't in the boundary. It is the right tool for what it was built for: key handling, tokenization or redaction inside an isolated CPU environment, including small models on CPU, with KMS policies tied to enclave measurements. ETH's finding that CPU TEEs can be cost-effective gives that route some support for small models at low batch sizes, though not for anything needing GPU throughput.

If you are an AWS shop that must have attested GPUs, your realistic options are running that one workload on Azure or buying from a specialist. Tinfoil offers inference on NVIDIA confidential hardware with client-side verification and open-source security-critical components. Its pricing page describes pay-as-you-go API pricing per model and custom enterprise quotes. Its technology page doesn't say who operates the machines, which matters after TEE.fail, so ask. Vendor continuity matters too, as OPAQUE's customers found out in October.

---

How to Decide: The Criteria That Predict Regret

Four questions settle this, and your cloud preference should come after them.

1. Who is the adversary, in writing? If it isn't the cloud operator, a data partner's requirement or a model-weights owner, stop here and use the table above.
2. Who consumes the evidence? If an external party will verify it, get their acceptance criteria before you build.
3. Can the workload tolerate Spot? Yes means Google is in play. No means Azure, or Google's G4 if an RTX PRO 6000 fits your model and you can price the licence.
4. Does your model fit one GPU? If it doesn't, this guide's generally available options all fail, and you are in an account-team conversation about Blackwell.

What changes the answer: AWS shipping an NVIDIA Confidential Computing instance, which would make a same-cloud deployment possible for its customers and is worth rechecking after re:Invent; Google adding on-demand or reservations to confidential A3; and Azure publishing a Blackwell confidential size. We found none of the three announced as of November 2, 2026.

---

What to Do About It

This Week:

1. Write the threat model on one page, naming the adversary and the party who will read the evidence. Send it to your security lead and your data partner's contact.
2. Trace one real request from client to log store and mark every point where plaintext exists. If retrieval, gateway or telemetry sit outside the proposed boundary, that is your first finding.

This Month:

1. Run your own model on one Standard_NCC40ads_H100_v5 and one standard NC40ads in East US 2, and record tokens per second, time to first token and model load time. Published overhead numbers range widely and come from specific serving stacks and models.
2. Build the release policy: Azure Attestation or your verifier for the CPU, nvattest or NRAS for the GPU, and a key that releases only if both pass. Test what happens when GPU attestation fails.
3. Set --max-log-len and block DEBUG in production for your serving layer.

Before Your Next Security Review:

1. Decide how you will answer the TDXRay and TEE.fail questions, because a competent assessor will ask. Record where your hardware physically sits and which risks you accept.
2. Set a calendar check for after re:Invent to reread AWS's GPU position and Google's confidential A3 provisioning options.

The Bottom Line

Confidential GPUs today resemble early customer-managed encryption keys: a real control with a narrow threat model. They help most when a named outside party needs proof, and they leave logs, gateways and indexes for you to secure separately.

Start with Azure's $6.98 NCC40ads in East US 2 if you have that outside party, and run the benchmark on your own model before you sign anything.

Continue Reading

• Together AI and Bedrock beat self-hosted gpt-oss-120b on cost
• Google Sensitive Data Protection Costs About $40 for 10M Chats
• Anthropic Hires Part of OPAQUE Team in Technology License Deal
• vLLM 0.30.0 Can Run Code Through Ghostscript Images
• Run vLLM on Bare Metal to Skip NVIDIA AI Enterprise Fees
• Samsung Led Mistral's Round. Score the Clauses, Not the Flag.