NVIDIA H100 NVL, AI & Machine Learning Benchmarks & Specs

94GB · AI Score 67.0/100 · anchored estimate vs 51 measured cards

58.1 AI Score Includes estimates

We have not run NVIDIA H100 NVL on our bench. These figures are anchored estimates, interpolated per workload against the 51 GPUs we did measure (confidence: high (sibling silicon)). On Llama 3.1 8B (Q4_K_M) NVIDIA H100 NVL should deliver about 307.8 tokens/sec. Stepping up to Qwen3 32B it should hold roughly 87.1 tok/s. The full Llama 3.3 70B still runs, at about 48.2 tok/s. For image generation, SDXL should run near 17.29 it/s, and FLUX.1-dev at 4.23 it/s. All 12 workloads fit in 94GB. There is no model in our suite this card has to turn down. NVIDIA H100 NVL isn't a retail purchase for most people. It's rented by the hour. You can run this exact card on RunPod.

AI & Machine Learning benchmark results

Text Generation tok/s 10

gpt-oss-20b296.75

Qwen3-4B282.91

Qwen3 30B A3B270.25

Llama-3.1-8B231.86

Qwen3 8B221.81

Gemma 4 12B135.74

Qwen3 14B135.54

Qwen2.5-Coder-14B125.63

Qwen3-32B64.06

Llama-3.3-70B32.79

Workload	Result	Telemetry	Data
Qwen3-4B	282.91 tok/s	177 W40°CQ4_K_M	✓ Measured
Llama-3.1-8B	231.86 tok/s	218 W43°CQ4_K_M	✓ Measured
Qwen3 8B	221.81 tok/s	221 W44°CQ4_K_M	✓ Measured
Gemma 4 12B	135.74 tok/s	229 W45°CQ4_K_M	✓ Measured
Qwen2.5-Coder-14B	125.63 tok/s	258 W47°CQ4_K_M	✓ Measured
Qwen3 14B	135.54 tok/s	265 W49°CQ4_K_M	✓ Measured
gpt-oss-20b	296.75 tok/s	162 W45°CQ4_K_M	✓ Measured
Qwen3 30B A3B	270.25 tok/s	151 W45°CQ4_K_M	✓ Measured
Qwen3-32B	64.06 tok/s	301 W51°CQ4_K_M	✓ Measured
Llama-3.3-70B	32.79 tok/s	310 W56°CQ4_K_M	✓ Measured

Image Generation images/min 3

Stable Diffusion XL34.58

Z-Image Turbo22.125

FLUX.1 dev9.064

Workload	Result	Telemetry	Data
Stable Diffusion XL	34.58 images/min	estimated	Est.
FLUX.1 dev	9.06 images/min	estimated	Est.
Z-Image Turbo	22.13 images/min	estimated	Est.

Image Editing images/min 2

Workload	Result	Telemetry	Data
FLUX.1 Kontext dev	4.26 images/min	estimated	Est.
Qwen-Image-Edit	3.68 images/min	estimated	Est.

Video Generation frames/s 2

Workload	Result	Telemetry	Data
LTX-Video (distilled)	17.47 frames/s	estimated	Est.
Wan 2.2 5B (720p)	1.33 frames/s	estimated	Est.

How this estimate is derived. This card hasn’t been through our bench yet, so its numbers are anchored estimates, interpolated from the 51 first-party measured cards (Sibling to measured H100 SXM: LLM ×1.176 (HBM3 3.9 vs 3.35 TB/s), diffusion ×1.0 (same 16896 cores); 94GB per GPU). The VRAM “won’t fit” gates are exact, since they’re pure capacity limits. Confidence: high (sibling silicon). Estimates are replaced with measured data as more silicon goes through the bench. Full methodology →

NVIDIA H100 NVL specifications

Architecture	Hopper
CUDA cores	16,896
VRAM	94GB HBM3
Memory bus	6016-bit
Memory bandwidth	3938 GB/s
Boost clock	1,785 MHz
TDP	400 W
Process	TSMC 4N
Interface	PCIe 5.0 x16
Release date	2023-03-21
Launch MSRP	$29,000

Verdict, nothing in our suite slows it down

NVIDIA H100 NVL scores 67.0/100, #8 of 102. It ran all 12 workloads. Figures are anchored estimates, not measurements, we flag that on every row.

Relative performance: where the NVIDIA H100 NVL lands

100% = this card, AI & Machine Learning headline metric (AI Score). #8 of 21 datacenter cards in this vertical.

GPU	%	AI Score
NVIDIA GH200 Grace Hopper	113%	65.6
NVIDIA H200	112%	65
NVIDIA H100 80GB HBM3	108%	62.6
NVIDIA H800 80GB	108%	62.6
NVIDIA H100 NVL	100%	58.1
NVIDIA RTX PRO 6000 Blackwell Server Edition	86%	49.9
NVIDIA H100 PCIe	86%	49.8
NVIDIA A100 80GB SXM4	57%	33.1
NVIDIA A800 80GB	57%	33.1

One tier up: NVIDIA H800 80GB (+8%)
One tier down: NVIDIA RTX PRO 6000 Blackwell Server Edition (-14%) · compare head-to-head

← All AI & Machine Learning GPU rankings

The silicon

Transistors	80,000 million
Die size	814 mm²
Process node	4 nm
Fabricated by	TSMC
Transistor density	98.3 million per mm²

Denser than 90% of the 746 cards we have silicon data for. Density is the clearest measure of what a process node bought: a card that gained it without growing the die got its speed from the fab rather than the architecture.

Silicon figures from Wikipedia (CC BY-SA 4.0). Benchmarks on this page are our own. Compare every chip.

What this card can build

Whole-job timings, composed from our measured per-model results on this card.

Workflow	Time	Energy	Basis
24-frame storyboard	89 s	1.82 Wh	estimate, 1 of 2 stages measured
60-second AI short film	2.1 min	1.17 Wh	estimate, 1 of 3 stages measured
6-panel comic page	2.3 min	0.91 Wh	estimate, 1 of 3 stages measured
Character sheet, 12 poses	2.9 min	n/a	estimate, 0 of 2 stages measured
Short social clips	6.8 min	0.65 Wh	estimate, 1 of 3 stages measured
Full codebase review	8 min	34.25 Wh	measured
Product photo shoot	10.5 min	n/a	estimate, 0 of 2 stages measured
Long-form article batch	14.3 min	73.56 Wh	measured
Photo restoration batch	23.5 min	n/a	estimate, 0 of 1 stage measured

Rent or buy?

This card is $29,000 to buy. The cheapest listed rate on Vast.ai is $2.469/hour, but that is the floor: we budget $2.963/hour, a 20% premium, because idle time, storage and unavailable cheap instances all land on the same bill. At that rate buying wins after 9,788 GPU-hours. Below it you are paying for idle silicon.

How you would use it	GPU-hours a year	Rental cost a year	Time to break even
2 hours a day, hobby	730	$2,163	13.4 years
8 hours a day, working on it	2,920	$8,651	3.4 years
24/7, always-on agent	8,760	$25,954	1.1 years

At hobby usage this card is very unlikely to pay for itself before it is superseded. Rent it. Rental figures include a 20% premium over the cheapest listed rate. Ignores electricity, resale and the fact that a rented card can be a newer one tomorrow.

Rental price

$2.590/hr+0.0% since 2026-08-14low $2.321 · high $2.590

Cheapest of the RunPod and Vast on-demand rates we see, sampled daily. Spot and interruptible pricing runs lower.