NVIDIA H100 NVL, AI & Machine Learning Benchmarks & Specs

94GB · AI Score 67.0/100 · anchored estimate vs 51 measured cards

67 AI Score Includes estimates

We have not run NVIDIA H100 NVL on our bench. These figures are anchored estimates, interpolated per workload against the 51 GPUs we did measure (confidence: high (sibling silicon)). On Llama 3.1 8B (Q4_K_M) NVIDIA H100 NVL should deliver about 307.8 tokens/sec. Stepping up to Qwen3 32B it should hold roughly 87.1 tok/s. The full Llama 3.3 70B still runs, at about 48.2 tok/s. For image generation, SDXL should run near 17.29 it/s, and FLUX.1-dev at 4.23 it/s. All 12 workloads fit in 94GB. There is no model in our suite this card has to turn down. NVIDIA H100 NVL isn't a retail purchase for most people. It's rented by the hour. You can run this exact card on RunPod.

AI & Machine Learning benchmark results

Text Generation tok/s 5

Qwen3 4B364.7
Llama 3.1 8B307.8
Qwen2.5-Coder 14B170.3
Qwen3 32B87.1
Llama 3.3 70B48.2
WorkloadResultTelemetryData
Qwen3 4B364.7 tok/sestimatedEst.
Llama 3.1 8B307.8 tok/sestimatedEst.
Qwen2.5-Coder 14B170.3 tok/sestimatedEst.
Qwen3 32B87.1 tok/sestimatedEst.
Llama 3.3 70B48.2 tok/sestimatedEst.

Image Generation images/min 3

Stable Diffusion XL34.58
Z-Image Turbo22.125
FLUX.1 dev9.064
WorkloadResultTelemetryData
Stable Diffusion XL34.58 images/minestimatedEst.
Z-Image Turbo22.13 images/minestimatedEst.
FLUX.1 dev9.06 images/minestimatedEst.

Image Editing images/min 2

WorkloadResultTelemetryData
FLUX.1 Kontext dev4.26 images/minestimatedEst.
Qwen-Image-Edit3.68 images/minestimatedEst.

Video Generation frames/s 2

WorkloadResultTelemetryData
LTX-Video (distilled)17.47 frames/sestimatedEst.
Wan 2.2 5B (720p)1.33 frames/sestimatedEst.
How this estimate is derived. This card hasn’t been through our bench yet, so its numbers are anchored estimates, interpolated from the 51 first-party measured cards (Sibling to measured H100 SXM: LLM ×1.176 (HBM3 3.9 vs 3.35 TB/s), diffusion ×1.0 (same 16896 cores); 94GB per GPU). The VRAM “won’t fit” gates are exact, since they’re pure capacity limits. Confidence: high (sibling silicon). Estimates are replaced with measured data as more silicon goes through the bench. Full methodology →

NVIDIA H100 NVL specifications

ArchitectureHopper
CUDA cores16,896
VRAM94GB HBM3
Memory bus6016-bit
Memory bandwidth3938 GB/s
Boost clock1,785 MHz
TDP400 W
ProcessTSMC 4N
InterfacePCIe 5.0 x16
Release date2023-03-21
Launch MSRP$29,000

Verdict, nothing in our suite slows it down

NVIDIA H100 NVL scores 67.0/100, #4 of 102. It ran all 12 workloads. Figures are anchored estimates, not measurements, we flag that on every row.

Relative performance: where the NVIDIA H100 NVL lands

100% = this card, AI & Machine Learning headline metric (AI Score). #4 of 21 desktop cards in this vertical.

GPURelative%AI Score
NVIDIA B300
140%93.8
NVIDIA B200
116%78
NVIDIA B100
100%67
NVIDIA H100 NVL
100%67
NVIDIA GH200 Grace Hopper
98%65.6
NVIDIA H200
97%65
NVIDIA H100 80GB HBM3
93%62.6
NVIDIA H800 80GB
93%62.6

← All AI & Machine Learning GPU rankings

What this card can build

Whole-job timings, composed from our measured per-model results on this card.

WorkflowTimeEnergyBasis
24-frame storyboard81 sn/aestimate, 0 of 2 stages measured
60-second AI short film2 minn/aestimate, 0 of 3 stages measured
6-panel comic page2.2 minn/aestimate, 0 of 3 stages measured
Character sheet, 12 poses2.9 minn/aestimate, 0 of 2 stages measured
Full codebase review5.9 minn/aestimate, 0 of 1 stage measured
10 short social clips6.7 minn/aestimate, 0 of 3 stages measured
20 long-form articles9.7 minn/aestimate, 0 of 1 stage measured
40-product photo shoot10.5 minn/aestimate, 0 of 2 stages measured
100-photo restoration batch23.5 minn/aestimate, 0 of 1 stage measured

Rent or buy?

This card is $29,000 to buy. The cheapest listed rate on RunPod is $2.590/hour, but that is the floor: we budget $3.108/hour, a 20% premium, because idle time, storage and unavailable cheap instances all land on the same bill. At that rate buying wins after 9,331 GPU-hours. Below it you are paying for idle silicon.

How you would use itGPU-hours a yearRental cost a yearTime to break even
2 hours a day, hobby730$2,26912.8 years
8 hours a day, working on it2,920$9,0753.2 years
24/7, always-on agent8,760$27,2261.1 years

At hobby usage this card is very unlikely to pay for itself before it is superseded. Rent it. Rental figures include a 20% premium over the cheapest listed rate. Ignores electricity, resale and the fact that a rented card can be a newer one tomorrow.

Rental price

$2.590/hr+0.0% since 2026-08-14low $2.590 · high $2.590

Cheapest of the RunPod and Vast on-demand rates we see, sampled daily. Spot and interruptible pricing runs lower.