NVIDIA A100 80GB SXM4, AI & Machine Learning Benchmarks & Specs

80GB · AI Score 33.1/100 · first-party measured on 12 AI workloads

33.1 AI Score ✓ Measured

Every number on this page is first-party: NVIDIA A100 80GB SXM4 was run on our pinned 12-workload AI suite on 2026-07-10, with under 0.5% run-to-run variance. On Llama 3.1 8B (Q4_K_M) NVIDIA A100 80GB SXM4 delivers about 162.57 tokens/sec. Stepping up to Qwen3 32B it holds roughly 45.53 tok/s. The full Llama 3.3 70B still runs, at about 24.4 tok/s. For image generation, SDXL runs at 8.18 it/s, and FLUX.1-dev at 1.99 it/s. All 12 workloads fit in 80GB. There is no model in our suite this card has to turn down. NVIDIA A100 80GB SXM4 isn't a retail purchase for most people. It's rented by the hour. You can run this exact card on RunPod.

AI & Machine Learning benchmark results

Text Generation tok/s 123

gemma-3-270m617.58

LFM2.5-1.2B576.95

Qwen2.5-0.5B571.49

Qwen2.5-Coder-0.5B562.59

SmolLM2-135M554.44

Qwen1.5-0.5B540.32

Llama 3.2 1B525.32

Qwen3-0.6B432.66

Qwen3 0.6B428.39

LFM2.5-8B-A1B369.01

Qwen3-1.7B350.4

Qwen3 1.7B343.58

Workload	Result	Telemetry	Data
gemma-3-270m	617.58 tok/s	103 W35°CQ4_K_M	✓ Measured
LFM2.5-1.2B	576.95 tok/s	142 W44°CQ4_K_M	✓ Measured
Qwen2.5-0.5B	571.49 tok/s	105 W35°CQ4_K_M	✓ Measured
Qwen2.5-Coder-0.5B	562.59 tok/s	109 W35°CQ4_K_M	✓ Measured
SmolLM2-135M	554.44 tok/s	104 W36°CQ4_K_M	✓ Measured
Qwen1.5-0.5B	540.32 tok/s	110 W37°CQ4_K_M	✓ Measured
Llama 3.2 1B	525.32 tok/s	99 W48°CQ4_K_M	✓ Measured
Qwen3-0.6B	432.66 tok/s	105 W36°CQ4_K_M	✓ Measured
Qwen3 0.6B	428.39 tok/s	96 W44°CQ4_K_M	✓ Measured
LFM2.5-8B-A1B	369.01 tok/s	146 W39°CQ4_K_M	✓ Measured
Qwen3-1.7B	350.4 tok/s	146 W39°CQ4_K_M	✓ Measured
Qwen3 1.7B	343.58 tok/s	102 W48°CQ4_K_M	✓ Measured
Qwen2.5-1.5B	335.29 tok/s	132 W37°CQ4_K_M	✓ Measured
Qwen2.5-Coder-1.5B	333.87 tok/s	131 W41°CQ4_K_M	✓ Measured
Qwen2-1.5B	333.79 tok/s	129 W39°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 1.5B	327.66 tok/s	112 W48°CQ4_K_M	✓ Measured
gemma-3-1b	324.08 tok/s	129 W37°CQ4_K_M	✓ Measured
Hermes-3-Llama-3.2-3B	269.13 tok/s	179 W42°CQ4_K_M	✓ Measured
Llama-3.2-3B-Instruct-uncensored	269.13 tok/s	138 W42°CQ4_K_M	✓ Measured
Llama 3.2 3B	264.25 tok/s	131 W51°CQ4_K_M	✓ Measured
SmolLM3-3B	254.6 tok/s	161 W41°CQ4_K_M	✓ Measured
SmolLM3 3B	249.76 tok/s	122 W51°CQ4_K_M	✓ Measured
Qwen2.5-Coder-3B	248.7 tok/s	170 W43°CQ4_K_M	✓ Measured
Qwen2.5-3B	248.47 tok/s	151 W40°CQ4_K_M	✓ Measured
gemma-2-2b	244.52 tok/s	149 W40°CQ4_K_M	✓ Measured
gemma-2-2b-it-abliterated	244.41 tok/s	159 W41°CQ4_K_M	✓ Measured
Phi-4-mini	243.38 tok/s	178 W43°CQ4_K_M	✓ Measured
AI21-Jamba-Reasoning-3B	239.23 tok/s	176 W41°CQ4_K_M	✓ Measured
Phi-4 Mini 3.8B	239.17 tok/s	126 W53°CQ4_K_M	✓ Measured
phi-2	235.55 tok/s	161 W40°CQ4_K_M	✓ Measured
Phi-3.5-mini	229.46 tok/s	209 W41°CQ4_K_M	✓ Measured
DeepSeek-Coder-V2-Lite	213.91 tok/s	150 W41°CQ4_K_M	✓ Measured
gpt-oss-20b	212.63 tok/s	154 W40°CQ4_K_M	✓ Measured
Nemotron-3-Nano-30B-A3B	204.06 tok/s	132 W43°CQ4_K_M	✓ Measured
Qwen3-4B-Thinking-2507	199.71 tok/s	184 W41°CQ4_K_M	✓ Measured
Qwen3-4B-Instruct-2507	199.65 tok/s	152 W42°CQ4_K_M	✓ Measured
Qwen3-4B-Instruct-2507	199.39 tok/s	155 W42°CQ4_K_M	✓ Measured
Qwen3 4B	197 tok/s	3.2 GB peak145 W50°C1.36 tok/WQ4_K_M	✓ Measured
Qwen3-Coder 30B A3B	182.26 tok/s	109 W51°CQ4_K_M	✓ Measured
Llama-2-7B	180.09 tok/s	226 W47°CQ4_K_M	✓ Measured
Qwen3-30B-A3B	178.14 tok/s	146 W39°CQ4_K_M	✓ Measured
Qwen3 30B A3B	177.32 tok/s	101 W53°CQ4_K_M	✓ Measured
Gemma 3 4B	176.94 tok/s	122 W50°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.1	173.87 tok/s	224 W45°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.2	173.8 tok/s	216 W45°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.3	173.27 tok/s	219 W44°CQ4_K_M	✓ Measured
Mistral 7B v0.3	171.92 tok/s	128 W55°CQ4_K_M	✓ Measured
Qwen2.5-7B	166.92 tok/s	235 W45°CQ4_K_M	✓ Measured
Qwen2.5-Coder-7B-Instruct-abliterated	166.84 tok/s	203 W44°CQ4_K_M	✓ Measured
Qwen2.5-Coder 7B	164.6 tok/s	114 W55°CQ4_K_M	✓ Measured
L3-8B-Stheno-v3.2	164.51 tok/s	200 W45°CQ4_K_M	✓ Measured
DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored	164.46 tok/s	230 W48°CQ4_K_M	✓ Measured
Meta-Llama-3.1-8B	164.24 tok/s	222 W44°CQ4_K_M	✓ Measured
dolphin-2.9-llama3-8b	164.09 tok/s	225 W45°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 7B	162.96 tok/s	153 W50°CQ4_K_M	✓ Measured
Llama 3.1 8B	162.57 tok/s	5.1 GB peak182 W55°C0.89 tok/WQ4_K_M	✓ Measured
DeepSeek-R1 Distill Llama 8B	159.37 tok/s	119 W51°CQ4_K_M	✓ Measured
Dolphin 3.0 Llama 3.1 8B	158.63 tok/s	174 W55°CQ4_K_M	✓ Measured
Dolphin X1 8B	158.62 tok/s	156 W55°CQ4_K_M	✓ Measured
Dolphin X1 Trinity Nano 6B	158.41 tok/s	131 W34°CQ4_K_M	✓ Measured
Qwen3-8B	152.8 tok/s	223 W45°CQ4_K_M	✓ Measured
DeepSeek-R1-0528-Qwen3-8B	152.76 tok/s	220 W44°CQ4_K_M	✓ Measured
Josiefied-Qwen3-8B-abliterated-v1	152.29 tok/s	219 W44°CQ4_K_M	✓ Measured
Qwen3 8B	149.96 tok/s	153 W48°CQ4_K_M	✓ Measured
Qwen3 30B A3B (Q3_K_M)	146.72 tok/s	149 W40°CQ3_K_M	✓ Measured
KAT-Coder-V2.5-Dev	146.38 tok/s	144 W39°CQ4_K_M	✓ Measured
Qwen-AgentWorld-35B-A3B	139.69 tok/s	141 W38°CQ4_K_M	✓ Measured
Ornith-1.0-35B	138.8 tok/s	135 W39°CQ4_K_M	✓ Measured
Ornith-1.0-9B	134.67 tok/s	193 W45°CQ4_K_M	✓ Measured
GLM-4.7-Flash	124.08 tok/s	146 W38°CQ4_K_M	✓ Measured
Qwen3-Next-80B-A3B-Thinking	120.98 tok/s	128 W40°CQ4_K_M	✓ Measured
Qwen3-Coder-Next	120.71 tok/s	130 W39°CQ4_K_M	✓ Measured
Qwen3-Coder-Next-abliterated	119.56 tok/s	129 W40°CQ4_K_M	✓ Measured
Qwen3-Next-80B-A3B-Thinking	118.93 tok/s	127 W40°CQ4_K_M	✓ Measured
Qwen3-Coder-Next	118.62 tok/s	127 W39°CQ4_K_M	✓ Measured
Qwen3-Next-80B-A3B	117.44 tok/s	126 W40°CQ4_K_M	✓ Measured
GLM-4.7-Flash-REAP-23B-A3B	112.75 tok/s	144 W38°CQ4_K_M	✓ Measured
Mistral-Nemo-Instruct-2407	111.07 tok/s	230 W46°CQ4_K_M	✓ Measured
NemoMix-Unleashed-12B	110.99 tok/s	236 W46°CQ4_K_M	✓ Measured
gemma-2-9b	108.68 tok/s	221 W44°CQ4_K_M	✓ Measured
Phi-4 14B	95.22 tok/s	173 W57°CQ4_K_M	✓ Measured
Hermes-4-14B	94.48 tok/s	244 W47°CQ4_K_M	✓ Measured
Qwen3-14B	94.35 tok/s	245 W46°CQ4_K_M	✓ Measured
Gemma 4 12B	92.43 tok/s	167 W56°CQ4_K_M	✓ Measured
Gemma 3 12B	91.45 tok/s	159 W53°CQ4_K_M	✓ Measured
Qwen3 14B	90.64 tok/s	163 W54°CQ4_K_M	✓ Measured
Qwen2.5-14B	90.46 tok/s	247 W46°CQ4_K_M	✓ Measured
EVA-Qwen2.5-14B-v0.2	90.35 tok/s	242 W47°CQ4_K_M	✓ Measured
Qwen2.5-Coder-14B-Instruct-abliterated	90.28 tok/s	231 W46°CQ4_K_M	✓ Measured
Qwen2.5-Coder 14B	89.45 tok/s	8.8 GB peak175 W56°C0.51 tok/WQ4_K_M	✓ Measured
Uncensored	89.43 tok/s	211 W46°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 14B	87 tok/s	149 W57°CQ4_K_M	✓ Measured
Phi-4 14B (Q3_K_M)	82.68 tok/s	251 W48°CQ3_K_M	✓ Measured
StarCoder2 15B	79.22 tok/s	178 W54°CQ4_K_M	✓ Measured
Gemma 3 12B (Q3_K_M)	71.88 tok/s	234 W45°CQ3_K_M	✓ Measured
DeepSeek-R1 Distill 14B (Q3_K_M)	67.13 tok/s	235 W46°CQ3_K_M	✓ Measured
Cydonia-24B-v4.3	65.82 tok/s	255 W48°CQ4_K_M	✓ Measured
Dolphin-Mistral-24B-Venice-Edition	65.77 tok/s	263 W48°CQ4_K_M	✓ Measured
Codestral 22B	63.26 tok/s	174 W53°CQ4_K_M	✓ Measured
Dolphin Mistral 24B Venice	62.72 tok/s	161 W50°CQ4_K_M	✓ Measured
Devstral Small 24B	61.63 tok/s	186 W51°CQ4_K_M	✓ Measured
Mistral Small 24B	61.38 tok/s	154 W58°CQ4_K_M	✓ Measured
Dolphin 3.0 R1 Mistral 24B	61.17 tok/s	187 W56°CQ4_K_M	✓ Measured
Codestral 22B (Q3_K_M)	49.52 tok/s	273 W49°CQ3_K_M	✓ Measured
Gemma 3 27B	48.43 tok/s	169 W60°CQ4_K_M	✓ Measured
Mistral Small 24B (Q3_K_M)	46.63 tok/s	259 W48°CQ3_K_M	✓ Measured
Olmo-3.1-32B-Think	46.49 tok/s	261 W50°CQ4_K_M	✓ Measured
Qwen2.5-32B	45.6 tok/s	266 W49°CQ4_K_M	✓ Measured
Qwen3 32B	45.53 tok/s	18.9 GB peak158 W59°C0.29 tok/WQ4_K_M	✓ Measured
DeepSeek-R1-Distill-Qwen-32B-abliterated	43.48 tok/s	233 W47°CQ4_K_M	✓ Measured
Dolphin 2.9.1 Yi 1.5 34B	43.47 tok/s	164 W54°CQ4_K_M	✓ Measured
Qwen2.5-Coder 32B	43.36 tok/s	174 W58°CQ4_K_M	✓ Measured
QwQ 32B	43.27 tok/s	160 W59°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 32B	43.11 tok/s	186 W61°CQ4_K_M	✓ Measured
Qwen2.5-Coder 32B (Q3_K_M)	31.99 tok/s	249 W50°CQ3_K_M	✓ Measured
Hermes-4-70B	24.47 tok/s	267 W52°CQ4_K_M	✓ Measured
Llama 3.3 70B	24.4 tok/s	40 GB peak175 W62°C0.14 tok/WQ4_K_M	✓ Measured
Qwen2.5-72B	23.95 tok/s	268 W52°CQ4_K_M	✓ Measured
DeepSeek-R1-Distill-Llama-70B	22.99 tok/s	229 W48°CQ4_K_M	✓ Measured
Llama-3.3-70B-Instruct-abliterated	22.92 tok/s	231 W50°CQ4_K_M	✓ Measured
Meta-Llama-3.1-70B	22.92 tok/s	235 W50°CQ4_K_M	✓ Measured
Laguna-XS-2.1	✕ Won't fit needs ~26 GB	VRAM-gated at this precision	✓ Measured
Nanbeige4.2-3B	✕ Won't fit needs ~4 GB	VRAM-gated at this precision	✓ Measured

Image Generation images/min 6

FLUX.1 Schnell28.39

Z-Image Turbo (1024px)18.65

Stable Diffusion XL16.36

Z-Image Turbo9.975

FLUX.1 dev4.264

Krea 2 Turbo3.35

Workload	Result	Telemetry	Data
FLUX.1 Schnell	28.39 images/min	394 W65°C	✓ Measured
Z-Image Turbo (1024px)	18.65 images/min	396 W68°C	✓ Measured
Stable Diffusion XL	16.36 images/min	14.7 GB peak370 W61°C3.7 s/img	✓ Measured
Z-Image Turbo	9.98 images/min	25.8 GB peak382 W67°C6 s/img	✓ Measured
FLUX.1 dev	4.26 images/min	36.7 GB peak396 W71°C14 s/img	✓ Measured
Krea 2 Turbo	3.35 images/min	346 W63°C	✓ Measured

Fine-Tuning train tok/s 4

SmolLM2 1.7B LoRA9750.8

TinyLlama 1.1B LoRA9553.3

Qwen2.5 1.5B LoRA8252.8

Qwen2.5 7B LoRA3900.6

Workload	Result	Telemetry	Data
SmolLM2 1.7B LoRA	9750.8 train tok/s	326 W44°C	✓ Measured
TinyLlama 1.1B LoRA	9553.3 train tok/s	259 W39°C	✓ Measured
Qwen2.5 1.5B LoRA	8252.8 train tok/s	260 W40°C	✓ Measured
Qwen2.5 7B LoRA	3900.6 train tok/s	394 W49°C	✓ Measured

LLM Serving serve tok/s 4

TinyLlama 1.1B served5701.5

Qwen2.5 1.5B served4564.7

SmolLM2 1.7B served4351.4

Qwen2.5 7B served2187.2

Workload	Result	Telemetry	Data
TinyLlama 1.1B served	5701.5 serve tok/s	179 W39°C	✓ Measured
Qwen2.5 1.5B served	4564.7 serve tok/s	192 W41°C	✓ Measured
SmolLM2 1.7B served	4351.4 serve tok/s	205 W42°C	✓ Measured
Qwen2.5 7B served	2187.2 serve tok/s	276 W45°C	✓ Measured

Image to Video clips/min 3

Wan 2.2 TI2V-5B (image to video)2.156

Stable Video Diffusion XT1.536

CogVideoX-5B I2V0.468

Workload	Result	Telemetry	Data
Wan 2.2 TI2V-5B (image to video)	2.16 clips/min	455 W66°C	✓ Measured
Stable Video Diffusion XT	1.54 clips/min	431 W67°C	✓ Measured
CogVideoX-5B I2V	0.47 clips/min	458 W70°C	✓ Measured

Image Editing images/min 2

Workload	Result	Telemetry	Data
FLUX.1 Kontext dev	1.99 images/min	35.5 GB peak395 W73°C30 s/img	✓ Measured
Qwen-Image-Edit	1.64 images/min	60.2 GB peak393 W74°C36.4 s/img	✓ Measured

Video Generation frames/s 2

Workload	Result	Telemetry	Data
LTX-Video (distilled)	8.95 frames/s	60.1 GB peak379 W69°C10.8 s/clip	✓ Measured
Wan 2.2 5B (720p)	0.66 frames/s	60.3 GB peak392 W74°C74.2 s/clip	✓ Measured

Image to 3D assets/hour 2

Workload	Result	Telemetry	Data
TRELLIS Image-to-3D	488.7 assets/hour	225 W54°C	✓ Measured
TRELLIS.2 Image-to-3D (1536³ max quality)	32.7 assets/hour	292 W77°C	✓ Measured

Depth Estimation images/min 2

Workload	Result	Telemetry	Data
Depth Anything V2 Small	925.15 images/min	81 W50°C	✓ Measured
Depth Anything V2 Large	876.29 images/min	82 W50°C	✓ Measured

Segmentation images/min 2

Workload	Result	Telemetry	Data
SAM ViT-Base	744.79 images/min	80 W51°C	✓ Measured
SAM ViT-Huge	145.7 images/min	303 W58°C	✓ Measured

Vision Language images/min 2

Workload	Result	Telemetry	Data
Florence-2 Base	126.4 images/min	79 W34°C	✓ Measured
Florence-2 Large	70.94 images/min	82 W35°C	✓ Measured

Background Removal images/min 1

Workload	Result	Telemetry	Data
BiRefNet	916.67 images/min	83 W50°C	✓ Measured

Upscaling images/min 1

Workload	Result	Telemetry	Data
Swin2SR 4x Upscaler	28.72 images/min	184 W56°C	✓ Measured

Text to Speech x realtime 1

Workload	Result	Telemetry	Data
Kokoro TTS 82M	110.17 x realtime	72 W40°C	✓ Measured

Music Generation x realtime 1

Workload	Result	Telemetry	Data
MusicGen Small	0.74 x realtime	84 W40°C	✓ Measured

Speech to Text x realtime 1

Workload	Result	Telemetry	Data
Whisper large-v3	73.83 x realtime	151 W45°C	✓ Measured

How we measured this. Every result comes from our own pinned, reproducible AI suite, 12 workloads: the Qwen3-4B to Llama-70B LLM ladder (llama.cpp, Q4_K_M), SDXL / Z-Image / FLUX-dev generation, FLUX-Kontext / Qwen-Edit editing, and LTX / Wan video, run first-party on rented hardware with under 0.5% run-to-run variance. Peak VRAM, power draw, temperature and tokens-per-watt are captured per workload. “Won’t fit” rows are real data: where a model exceeds the card’s VRAM at the tested precision we record a hard gate rather than silently dropping to a smaller quant. Measured 2026-07-10 · harness 2.0.0.

NVIDIA A100 80GB SXM4 specifications

Architecture	Ampere
CUDA cores	6,912
VRAM	80GB HBM2e
Memory bus	5120-bit
Memory bandwidth	2039 GB/s
Boost clock	1,410 MHz
TDP	400 W
Process	TSMC 7nm
Interface	SXM4
Release date	2020-11-16
Launch MSRP	$17,000

Verdict, NVIDIA A100 80GB SXM4 on real AI workloads

NVIDIA A100 80GB SXM4 scores 33.1/100, #13 of 102. It ran all 12 workloads. Every figure here is our own measurement.

Relative performance: where the NVIDIA A100 80GB SXM4 lands

100% = this card, AI & Machine Learning headline metric (AI Score). #11 of 21 desktop cards in this vertical.

GPU	%	AI Score
NVIDIA H100 80GB HBM3	189%	62.6
NVIDIA H800 80GB	189%	62.6
NVIDIA RTX PRO 6000 Blackwell Server Edition	151%	49.9
NVIDIA H100 PCIe	140%	46.4
NVIDIA A100 80GB SXM4	100%	33.1
NVIDIA A800 80GB	100%	33.1
NVIDIA A100 80GB PCIe	96%	31.7
NVIDIA L40S	84%	27.7
NVIDIA L40	59%	19.5

One tier up: NVIDIA H100 PCIe (+40%)
One tier down: NVIDIA A800 80GB (0%)
Best value alternative ($15,000): NVIDIA A800 80GB (0%)

← All AI & Machine Learning GPU rankings

What this card can build

Whole-job timings, composed from our measured per-model results on this card.

Workflow	Time	Energy	Basis
50-image depth pass	5 s	0.08 Wh	measured
30-minute podcast pass	67 s	1.81 Wh	all 3 stages measured
500-image masking run	3.5 min	17.35 Wh	measured
24-frame storyboard	3.7 min	16.68 Wh	all 2 stages measured
20-asset 3D game kit	4 min	16.75 Wh	all 2 stages measured
60-second AI short film	4.7 min	23.61 Wh	all 3 stages measured
60-second AI short film, narrated	4.9 min	23.62 Wh	all 4 stages measured
6-panel comic page	6 min	29.8 Wh	all 3 stages measured
200-product catalogue cutout	7.3 min	21.66 Wh	all 2 stages measured
Character sheet, 12 poses	7.6 min	41.23 Wh	all 2 stages measured
Full codebase review	11.2 min	32.59 Wh	measured
10 short social clips	15.1 min	87.69 Wh	all 3 stages measured
20 long-form articles	19.1 min	55.72 Wh	measured
40-product photo shoot	23.5 min	147.33 Wh	all 2 stages measured
40-product shoot, start to finish	25 min	151.66 Wh	all 4 stages measured
100-photo restoration batch	50.8 min	330.66 Wh	measured
100-photo restore and enlarge	54.3 min	341.34 Wh	all 2 stages measured

Rent or buy?

This card is $17,000 to buy. The cheapest listed rate on RunPod is $1.390/hour, but that is the floor: we budget $1.668/hour, a 20% premium, because idle time, storage and unavailable cheap instances all land on the same bill. At that rate buying wins after 10,192 GPU-hours. Below it you are paying for idle silicon.

How you would use it	GPU-hours a year	Rental cost a year	Time to break even
2 hours a day, hobby	730	$1,218	14.0 years
8 hours a day, working on it	2,920	$4,871	3.5 years
24/7, always-on agent	8,760	$14,612	1.2 years

At hobby usage this card is very unlikely to pay for itself before it is superseded. Rent it. Rental figures include a 20% premium over the cheapest listed rate. Ignores electricity, resale and the fact that a rented card can be a newer one tomorrow.

Rental price

$1.390/hr+0.0% since 2026-08-14low $1.390 · high $1.390

Cheapest of the RunPod and Vast on-demand rates we see, sampled daily. Spot and interruptible pricing runs lower.