NVIDIA A100 40GB PCIe, AI & Machine Learning Benchmarks & Specs

40GB · AI Score 16.7/100 · anchored estimate vs 51 measured cards

16.7 AI Score Includes estimates

We have not run NVIDIA A100 40GB PCIe on our bench. These figures are anchored estimates, interpolated per workload against the 51 GPUs we did measure (confidence: high (sibling silicon)). On Llama 3.1 8B (Q4_K_M) NVIDIA A100 40GB PCIe should deliver about 130 tokens/sec. Stepping up to Qwen3 32B it should hold roughly 35.2 tok/s. Llama 3.3 70B does not fit. It needs roughly 42GB and this card has 40GB. For image generation, SDXL should run near 8.4 it/s, and FLUX.1-dev at 1.85 it/s. 2 of the 12 workloads won't fit on 40GB at the tested precision, Llama 3.3 70B, Qwen-Image-Edit. We publish those as hard gates rather than quietly dropping to a smaller quant.

AI & Machine Learning benchmark results

Text Generation tok/s 12

Granite 4.1 3B207.93

Qwen3-4B197.27

Qwen3 30B A3B182.96

Llama-3.1-8B154.22

Qwen3 8B146.86

Gemma 4 26B A4B136.74

Gemma 4 12B90.61

Qwen3 14B87.51

Qwen2.5-Coder-14B84.11

Gemma 4 31B42.57

Qwen3-32B41.5

Workload	Result	Telemetry	Data
Granite 4.1 3B	207.93 tok/s	136 W43°CQ4_K_M	✓ Measured
Qwen3-4B	197.27 tok/s	144 W43°CQ4_K_M	✓ Measured
Llama-3.1-8B	154.22 tok/s	162 W43°CQ4_K_M	✓ Measured
Qwen3 8B	146.86 tok/s	164 W44°CQ4_K_M	✓ Measured
Gemma 4 12B	90.61 tok/s	166 W46°CQ4_K_M	✓ Measured
Qwen2.5-Coder-14B	84.11 tok/s	177 W47°CQ4_K_M	✓ Measured
Qwen3 14B	87.51 tok/s	166 W46°CQ4_K_M	✓ Measured
Gemma 4 26B A4B	136.74 tok/s	109 W41°CQ4_K_M	✓ Measured
Qwen3 30B A3B	182.96 tok/s	114 W41°CQ4_K_M	✓ Measured
Gemma 4 31B	42.57 tok/s	183 W49°CQ4_K_M	✓ Measured
Qwen3-32B	41.5 tok/s	188 W48°CQ4_K_M	✓ Measured
Llama 3.3 70B	✕ Won't fit	VRAM-gated at this precision	Est.

Image Generation images/min 4

Stable Diffusion XL16.8

Z-Image Turbo9.3

FLUX.1 dev3.964

Chroma1-HD1.07

Workload	Result	Telemetry	Data
Stable Diffusion XL	16.8 images/min	estimated	Est.
Chroma1-HD	1.07 images/min	248 W70°C	✓ Measured
FLUX.1 dev	3.96 images/min	estimated	Est.
Z-Image Turbo	9.3 images/min	estimated	Est.

Image Editing images/min 2

Workload	Result	Telemetry	Data
FLUX.1 Kontext dev	1.84 images/min	estimated	Est.
Qwen-Image-Edit	✕ Won't fit	VRAM-gated at this precision	Est.

Video Generation frames/s 2

Workload	Result	Telemetry	Data
LTX-Video (distilled)	8.51 frames/s	estimated	Est.
Wan 2.2 5B (720p)	0.61 frames/s	estimated	Est.

How this estimate is derived. This card hasn’t been through our bench yet, so its numbers are anchored estimates, interpolated from the 51 first-party measured cards (Sibling-anchored to measured A100 80GB: LLM ×0.804 (bandwidth ratio), diffusion ×1.0 (identical compute); VRAM gates at 40GB). The VRAM “won’t fit” gates are exact, since they’re pure capacity limits. Confidence: high (sibling silicon). Estimates are replaced with measured data as more silicon goes through the bench. Full methodology →

NVIDIA A100 40GB PCIe specifications

Architecture	Ampere
CUDA cores	6,912
VRAM	40GB HBM2
Memory bus	5120-bit
Memory bandwidth	1555 GB/s
Boost clock	1,410 MHz
TDP	250 W
Process	TSMC 7nm
Interface	PCIe 4.0 x16
Release date	2020-06-22
Launch MSRP	$10,000

Verdict, capable, but 40GB sets the ceiling

NVIDIA A100 40GB PCIe scores 16.7/100, #24 of 102. It ran 10 of 12; 2 exceeded its 40GB. Figures are anchored estimates, not measurements, we flag that on every row.

Relative performance: where the NVIDIA A100 40GB PCIe lands

100% = this card, AI & Machine Learning headline metric (AI Score). #18 of 21 datacenter cards in this vertical.

GPU	%	AI Score
NVIDIA L40S	166%	27.7
NVIDIA L40	117%	19.5
NVIDIA A40	106%	17.7
NVIDIA A100 40GB SXM4	102%	17
NVIDIA A100 40GB PCIe	100%	16.7
NVIDIA A10G	38%	6.4
NVIDIA L4	30%	5
NVIDIA T4	17%	2.9

One tier up: NVIDIA A100 40GB SXM4 (+2%) · compare head-to-head
One tier down: NVIDIA A10G (-62%) · compare head-to-head
Best value alternative ($5,000): NVIDIA A40 (+6%) · compare head-to-head

← All AI & Machine Learning GPU rankings

The silicon

Transistors	54,200 million
Die size	826 mm²
Process node	7 nm
Fabricated by	TSMC
Transistor density	65.6 million per mm²

Denser than 88% of the 746 cards we have silicon data for. Density is the clearest measure of what a process node bought: a card that gained it without growing the die got its speed from the fab rather than the architecture.

Silicon figures from Wikipedia (CC BY-SA 4.0). Benchmarks on this page are our own. Compare every chip.

What this card can build

Whole-job timings, composed from our measured per-model results on this card.

Workflow	Time	Energy	Basis
24-frame storyboard	3.2 min	1.76 Wh	estimate, 1 of 2 stages measured
60-second AI short film	4.2 min	1.13 Wh	estimate, 1 of 3 stages measured
6-panel comic page	5.1 min	0.88 Wh	estimate, 1 of 3 stages measured
Character sheet, 12 poses	6.8 min	n/a	estimate, 0 of 2 stages measured
Full codebase review	11.9 min	35.07 Wh	measured
Short social clips	14.7 min	0.63 Wh	estimate, 1 of 3 stages measured
Product photo shoot	24.1 min	n/a	estimate, 0 of 2 stages measured
Photo restoration batch	54.3 min	n/a	estimate, 0 of 1 stage measured

Can't run: Long-form article batch (needs Llama 3.3 70B).

Rent or buy?

This card is $10,000 to buy. The cheapest listed rate on Vast.ai is $0.508/hour, but that is the floor: we budget $0.610/hour, a 20% premium, because idle time, storage and unavailable cheap instances all land on the same bill. At that rate buying wins after 16,404 GPU-hours. Below it you are paying for idle silicon.

How you would use it	GPU-hours a year	Rental cost a year	Time to break even
2 hours a day, hobby	730	$445	22.5 years
8 hours a day, working on it	2,920	$1,780	5.6 years
24/7, always-on agent	8,760	$5,340	1.9 years

At hobby usage this card is very unlikely to pay for itself before it is superseded. Rent it. Rental figures include a 20% premium over the cheapest listed rate. Ignores electricity, resale and the fact that a rented card can be a newer one tomorrow.

Rental price

$0.508/hr+25.7% since 2026-09-30low $0.341 · high $0.549

Cheapest of the RunPod and Vast on-demand rates we see, sampled daily. Spot and interruptible pricing runs lower.