NVIDIA L4, AI & Machine Learning Benchmarks & Specs

24GB · AI Score 5.0/100 · first-party measured on 12 AI workloads

5 AI Score ✓ Measured

Every number on this page is first-party: NVIDIA L4 was run on our pinned 12-workload AI suite on 2026-07-10, with under 0.5% run-to-run variance. On Llama 3.1 8B (Q4_K_M) NVIDIA L4 delivers about 50.45 tokens/sec. Llama 3.3 70B does not fit. It needs roughly 42GB and this card has 24GB. For image generation, SDXL runs at 2.59 it/s, while FLUX.1-dev won't fit at BF16 (needs ~26GB). 5 of the 12 workloads won't fit on 24GB at the tested precision, Qwen3 32B, Llama 3.3 70B, FLUX.1-dev, FLUX.1 Kontext and others. We publish those as hard gates rather than quietly dropping to a smaller quant. NVIDIA L4 isn't a retail purchase for most people. It's rented by the hour. You can run this exact card on RunPod.

AI & Machine Learning benchmark results

Text Generation tok/s 143

SmolLM2-135M652.15

gemma-3-270m-it493.31

SmolLM2-360M440.21

Qwen1.5-0.5B431.41

Qwen2.5-0.5B-Instruct399.4

Qwen2.5-Coder-0.5B397.1

Qwen2 0.5B396.94

Qwen3 0.6B357.79

Qwen_Qwen3-0.6B356.93

LFM2.5-1.2B278.53

Llama 3.2 1B259.48

gemma-3-1b-it216.27

Workload	Result	Telemetry	Data
SmolLM2-135M	652.15 tok/s	36 W63°CQ4_K_M	✓ Measured
gemma-3-270m-it	493.31 tok/s	34 W51°CQ4_K_M	✓ Measured
SmolLM2-360M	440.21 tok/s	34 W55°CQ4_K_M	✓ Measured
Qwen1.5-0.5B	431.41 tok/s	41 W64°CQ4_K_M	✓ Measured
Qwen2 0.5B	396.94 tok/s	38 W55°CQ4_K_M	✓ Measured
Qwen2.5-0.5B-Instruct	399.4 tok/s	41 W53°CQ4_K_M	✓ Measured
Qwen2.5-Coder-0.5B	397.1 tok/s	39 W63°CQ4_K_M	✓ Measured
Qwen3 0.6B	357.79 tok/s	46 W51°CQ4_K_M	✓ Measured
Qwen_Qwen3-0.6B	356.93 tok/s	40 W54°CQ4_K_M	✓ Measured
Llama 3.2 1B	259.48 tok/s	48 W54°CQ4_K_M	✓ Measured
gemma-3-1b-it	216.27 tok/s	50 W52°CQ4_K_M	✓ Measured
LFM2.5-1.2B	278.53 tok/s	46 W58°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 1.5B	191.38 tok/s	51 W53°CQ4_K_M	✓ Measured
Qwen2-1.5B	189.18 tok/s	52 W64°CQ4_K_M	✓ Measured
Qwen2.5-1.5B-Instruct	191.39 tok/s	53 W55°CQ4_K_M	✓ Measured
Qwen2.5-Coder-1.5B	184.6 tok/s	47 W58°CQ4_K_M	✓ Measured
Qwen3 1.7B	177.5 tok/s	51 W51°CQ4_K_M	✓ Measured
Qwen3-1.7B	176.08 tok/s	52 W61°CQ4_K_M	✓ Measured
MiniCPM5 2B	138.25 tok/s	52 W56°CQ4_K_M	✓ Measured
gemma-2-2b-it	114.17 tok/s	57 W53°CQ4_K_M	✓ Measured
gemma-2-2b-it-abliterated	113 tok/s	55 W63°CQ4_K_M	✓ Measured
LFM2.5 2.6B	128.88 tok/s	53 W56°CQ4_K_M	✓ Measured
AI21-Jamba-Reasoning-3B	107.59 tok/s	59 W64°CQ4_K_M	✓ Measured
Granite 4.1 3B	96.18 tok/s	55 W66°CQ4_K_M	✓ Measured
Hermes-3-Llama-3.2-3B	106.14 tok/s	56 W64°CQ4_K_M	✓ Measured
Llama 3.2 3B	107.1 tok/s	55 W53°CQ4_K_M	✓ Measured
Llama-3.2-3B-Instruct-uncensored	101.16 tok/s	56 W62°CQ4_K_M	✓ Measured
Nanbeige4.2-3B	✕ Won't fit needs ~4 GB	VRAM-gated at this precision	✓ Measured
Qwen2.5-3B-Instruct	110.84 tok/s	57 W55°CQ4_K_M	✓ Measured
Qwen2.5-Coder-3B	109.42 tok/s	54 W58°CQ4_K_M	✓ Measured
SmolLM3 3B	112.07 tok/s	57 W54°CQ4_K_M	✓ Measured
SmolLM3-3B	110.77 tok/s	53 W61°CQ4_K_M	✓ Measured
Phi-4 Mini 3.8B	88.48 tok/s	57 W54°CQ4_K_M	✓ Measured
Gemma 3 4B	82.03 tok/s	60 W54°CQ4_K_M	✓ Measured
Nemotron 3 Nano 4B	79.8 tok/s	58 W54°CQ4_K_M	✓ Measured
Qwen3-4B	84 tok/s	59 W55°CQ4_K_M	✓ Measured
Qwen3-4B-Instruct-2507	83.13 tok/s	60 W65°CQ4_K_M	✓ Measured
Qwen3-4B-Instruct-2507	83.03 tok/s	60 W66°CQ4_K_M	✓ Measured
Qwen3-4B-Thinking-2507	83.12 tok/s	59 W63°CQ4_K_M	✓ Measured
phi-2	114.99 tok/s	57 W63°CQ4_K_M	✓ Measured
Dolphin X1 Trinity Nano 6B	166.6 tok/s	47 W62°CQ4_K_M	✓ Measured
Phi-3.5-mini-instruct	90.56 tok/s	60 W51°CQ4_K_M	✓ Measured
Phi-4-mini	87.21 tok/s	56 W63°CQ4_K_M	✓ Measured
DeepSeek Coder 7B Instruct v1.5	57.06 tok/s	59 W66°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 7B	53.27 tok/s	63 W53°CQ4_K_M	✓ Measured
Llama-2-7B	56.06 tok/s	63 W62°CQ4_K_M	✓ Measured
Mistral 7B v0.3	54.02 tok/s	64 W54°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.1	53.6 tok/s	63 W64°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.2	53.31 tok/s	63 W59°CQ4_K_M	✓ Measured
Mistral-7B-Instruct-v0.3	53.66 tok/s	63 W62°CQ4_K_M	✓ Measured
Qwen2.5-7B-Instruct	53.32 tok/s	63 W55°CQ4_K_M	✓ Measured
Qwen2.5-Coder 7B	53.26 tok/s	63 W55°CQ4_K_M	✓ Measured
Qwen2.5-Coder-7B-Instruct-abliterated	53.02 tok/s	61 W64°CQ4_K_M	✓ Measured
Qwen2.5-VL 7B Instruct	52.57 tok/s	54 W65°CQ4_K_M	✓ Measured
DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored	48.82 tok/s	60 W64°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill Llama 8B	50.39 tok/s	62 W55°CQ4_K_M	✓ Measured
DeepSeek-R1-0528-Qwen3-8B	49.07 tok/s	63 W52°CQ4_K_M	✓ Measured
Dolphin 3.0 Llama 3.1 8B	50.24 tok/s	63 W64°CQ4_K_M	✓ Measured
Dolphin X1 8B	50.12 tok/s	63 W62°CQ4_K_M	✓ Measured
Josiefied-Qwen3-8B-abliterated-v1	48.72 tok/s	62 W64°CQ4_K_M	✓ Measured
L3-8B-Stheno-v3.2	50.07 tok/s	61 W65°CQ4_K_M	✓ Measured
LFM2.5-8B-A1B	170.2 tok/s	48 W63°CQ4_K_M	✓ Measured
Llama 3 8B	50.35 tok/s	59 W67°CQ4_K_M	✓ Measured
Llama 3.1 8B	50.45 tok/s	4.8 GB peak60 W50°C0.84 tok/WQ4_K_M	✓ Measured
Meta-Llama-3.1-8B-Instruct	50.36 tok/s	62 W57°CQ4_K_M	✓ Measured
Qwen3 8B	48.95 tok/s	63 W53°CQ4_K_M	✓ Measured
Qwen3-8B	48.34 tok/s	60 W64°CQ4_K_M	✓ Measured
dolphin-2.9-llama3-8b	50.04 tok/s	61 W66°CQ4_K_M	✓ Measured
Nemotron Nano 9B v2	35.8 tok/s	61 W67°CQ4_K_M	✓ Measured
Ornith 1.5 9B	43.76 tok/s	61 W56°CQ4_K_M	✓ Measured
Ornith-1.0-9B	43.35 tok/s	62 W53°CQ4_K_M	✓ Measured
gemma-2-9b	33.8 tok/s	64 W64°CQ4_K_M	✓ Measured
Gemma 3 12B	31.01 tok/s	65 W55°CQ4_K_M	✓ Measured
Gemma 3 12B (Q3_K_M)	32.29 tok/s	62 W62°CQ3_K_M	✓ Measured
Gemma 4 12B	31.49 tok/s	65 W56°CQ4_K_M	✓ Measured
NemoMix-Unleashed-12B	33.02 tok/s	63 W64°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 14B	27.39 tok/s	65 W59°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 14B (Q3_K_M)	27.87 tok/s	65 W64°CQ3_K_M	✓ Measured
EVA-Qwen2.5-14B-v0.2	27.3 tok/s	64 W63°CQ4_K_M	✓ Measured
Hermes-4-14B	27.48 tok/s	65 W65°CQ4_K_M	✓ Measured
Phi-4 14B	27.24 tok/s	66 W57°CQ4_K_M	✓ Measured
Phi-4 14B (Q3_K_M)	27.86 tok/s	64 W62°CQ3_K_M	✓ Measured
Qwen2.5-14B-Instruct	27.37 tok/s	65 W55°CQ4_K_M	✓ Measured
Qwen2.5-Coder 14B	27.46 tok/s	8.6 GB peak62 W57°C0.44 tok/WQ4_K_M	✓ Measured
Qwen2.5-Coder-14B-Instruct-abliterated	27.29 tok/s	65 W65°CQ4_K_M	✓ Measured
Qwen3 14B	27.61 tok/s	65 W55°CQ4_K_M	✓ Measured
Qwen3-14B	27.48 tok/s	60 W65°CQ4_K_M	✓ Measured
Uncensored	27.29 tok/s	65 W66°CQ4_K_M	✓ Measured
StarCoder2 15B	24.4 tok/s	66 W60°CQ4_K_M	✓ Measured
Mistral-Nemo-Instruct-2407	33.01 tok/s	63 W64°CQ4_K_M	✓ Measured
gpt-oss-20b	91.78 tok/s	54 W62°CQ4_K_M	✓ Measured
Codestral 22B	18.14 tok/s	67 W61°CQ4_K_M	✓ Measured
Codestral 22B (Q3_K_M)	17.96 tok/s	67 W68°CQ3_K_M	✓ Measured
GLM-4.7-Flash-REAP-23B-A3B	70.87 tok/s	54 W65°CQ4_K_M	✓ Measured
DeepSeek-Coder-V2-Lite	110.11 tok/s	51 W62°CQ4_K_M	✓ Measured
Cydonia-24B-v4.3	17.3 tok/s	66 W64°CQ4_K_M	✓ Measured
Devstral Small 24B	17.34 tok/s	66 W60°CQ4_K_M	✓ Measured
Dolphin 3.0 R1 Mistral 24B	17.32 tok/s	67 W63°CQ4_K_M	✓ Measured
Dolphin Mistral 24B Venice	17.34 tok/s	67 W63°CQ4_K_M	✓ Measured
Dolphin-Mistral-24B-Venice-Edition	17.2 tok/s	64 W64°CQ4_K_M	✓ Measured
Mistral Small 24B	17.33 tok/s	66 W61°CQ4_K_M	✓ Measured
Mistral Small 24B (Q3_K_M)	16.62 tok/s	67 W67°CQ3_K_M	✓ Measured
Gemma 4 26B A4B	68.51 tok/s	54 W54°CQ4_K_M	✓ Measured
Gemma 3 27B	14.18 tok/s	67 W63°CQ4_K_M	✓ Measured
Qwen3.6 27B	14.2 tok/s	66 W58°CQ4_K_M	✓ Measured
Qwen3.8 27B	13.91 tok/s	67 W58°CQ4_K_M	✓ Measured
Nemotron 3.5 Lightning 30B A3B	✕ Won't fit needs ~30 GB	VRAM-gated at this precision	✓ Measured
Nemotron-3-Nano-30B-A3B	✕ Won't fit needs ~31 GB	VRAM-gated at this precision	✓ Measured
Qwen3 30B A3B	96.15 tok/s	52 W57°CQ4_K_M	✓ Measured
Qwen3 30B A3B (Q3_K_M)	94.03 tok/s	52 W66°CQ3_K_M	✓ Measured
Qwen3 30B A3B Instruct 2507	99.74 tok/s	45 W70°CQ4_K_M	✓ Measured
Qwen3-30B-A3B	95.02 tok/s	51 W63°CQ4_K_M	✓ Measured
Qwen3-Coder 30B A3B	99.14 tok/s	53 W58°CQ4_K_M	✓ Measured
Gemma 4 31B	12.84 tok/s	67 W61°CQ4_K_M	✓ Measured
DeepSeek-R1 Distill 32B	12.28 tok/s	67 W63°CQ4_K_M	✓ Measured
DeepSeek-R1-Distill-Qwen-32B-abliterated	12.26 tok/s	66 W66°CQ4_K_M	✓ Measured
Olmo-3.1-32B-Think	12.27 tok/s	67 W63°CQ4_K_M	✓ Measured
QwQ 32B	12.28 tok/s	67 W63°CQ4_K_M	✓ Measured
Qwen2.5-32B	12.24 tok/s	65 W59°CQ4_K_M	✓ Measured
Qwen2.5-Coder 32B	12.29 tok/s	67 W62°CQ4_K_M	✓ Measured
Qwen2.5-Coder 32B (Q3_K_M)	12.05 tok/s	67 W68°CQ3_K_M	✓ Measured
Qwen3-32B	12.42 tok/s	67 W65°CQ4_K_M	✓ Measured
Dolphin 2.9.1 Yi 1.5 34B	11.92 tok/s	67 W65°CQ4_K_M	✓ Measured
Ornith 1.5 35B A3B	84.69 tok/s	46 W68°CQ4_K_M	✓ Measured
Ornith-1.0-35B	70.31 tok/s	52 W55°CQ4_K_M	✓ Measured
Qwen-AgentWorld-35B-A3B	70.23 tok/s	53 W57°CQ4_K_M	✓ Measured
Qwen3.6 35B A3B	78.98 tok/s	51 W55°CQ4_K_M	✓ Measured
GLM-4.7-Flash	75.03 tok/s	52 W53°CQ4_K_M	✓ Measured
Laguna-XS-2.1	✕ Won't fit needs ~26 GB	VRAM-gated at this precision	✓ Measured
Kwaipilot_KAT-Coder-V2.5-Dev	79.01 tok/s	51 W54°CQ4_K_M	✓ Measured
DeepSeek-R1-Distill-Llama-70B	✕ Won't fit needs ~54 GB	VRAM-gated at this precision	✓ Measured
Hermes-4-70B	✕ Won't fit needs ~54 GB	VRAM-gated at this precision	✓ Measured
Llama 3.3 70B	✕ Won't fit needs ~46 GB	VRAM-gated at this precision	✓ Measured
Llama-3.3-70B-Instruct-abliterated	✕ Won't fit needs ~54 GB	VRAM-gated at this precision	✓ Measured
Meta-Llama-3.1-70B	✕ Won't fit needs ~54 GB	VRAM-gated at this precision	✓ Measured
Qwen2.5-72B	✕ Won't fit needs ~60 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Next-80B-A3B	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Next-80B-A3B-Thinking	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Next-80B-A3B-Thinking	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Coder-Next	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Coder-Next	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
Qwen3-Coder-Next-abliterated	✕ Won't fit needs ~61 GB	VRAM-gated at this precision	✓ Measured
gpt-oss-120b	✕ Won't fit needs ~70 GB	VRAM-gated at this precision	✓ Measured

Embeddings sentences/s 1

Workload	Result	Telemetry	Data
BGE-Large Embeddings	1016.5 sentences/s	73 W58°C	✓ Measured

Image Generation images/min 15

SDXL Turbo229.86

Stable Diffusion 1.528.64

Sana 1.6B13.84

PixArt-Sigma XL8.08

Stable Diffusion XL5.18

Z-Image Turbo (1024px)4.6

Playground v2.53.01

Stable Diffusion 3.5 Medium2.96

Z-Image Turbo2.475

AuraFlow v0.31.02

Z-Image0.37

Workload	Result	Telemetry	Data
Stable Diffusion 1.5	28.64 images/min	72 W69°C	✓ Measured
SDXL Turbo	229.86 images/min	46 W68°C	✓ Measured
Sana 1.6B	13.84 images/min	72 W56°C	✓ Measured
Stable Diffusion XL	5.18 images/min	14.6 GB peak72 W65°C11.6 s/img	✓ Measured
Playground v2.5	3.01 images/min	72 W80°C	✓ Measured
PixArt-Sigma XL	8.08 images/min	72 W61°C	✓ Measured
Stable Diffusion 3.5 Medium	2.96 images/min	72 W71°C	✓ Measured
Z-Image Turbo	2.48 images/min	21.9 GB peak72 W74°C24.6 s/img	✓ Measured
Z-Image	0.37 images/min	73 W78°C	✓ Measured
Z-Image Turbo (1024px)	4.6 images/min	71 W81°C	✓ Measured
AuraFlow v0.3	1.02 images/min	72 W78°C	✓ Measured
FLUX.1 dev	✕ Won't fit needs ~26 GB	VRAM-gated at this precision	✓ Measured
Stable Diffusion 3.5 Large	✕ Won't fit needs ~30 GB	VRAM-gated at this precision	✓ Measured
FLUX.1 Schnell	✕ Won't fit needs ~35 GB	VRAM-gated at this precision	✓ Measured
Krea 2 Turbo	✕ Won't fit needs ~44 GB	VRAM-gated at this precision	✓ Measured

Image Editing images/min 2

Workload	Result	Telemetry	Data
FLUX.1 Kontext dev	✕ Won't fit needs ~26 GB	VRAM-gated at this precision	✓ Measured
Qwen-Image-Edit	✕ Won't fit needs ~42 GB	VRAM-gated at this precision	✓ Measured

Background Removal images/min 1

Workload	Result	Telemetry	Data
BiRefNet	305.81 images/min	46 W50°C	✓ Measured

Upscaling images/min 1

Workload	Result	Telemetry	Data
Swin2SR 4x Upscaler	18.23 images/min	72 W47°C	✓ Measured

Image to Video clips/min 5

Workload	Result	Telemetry	Data
Stable Video Diffusion	0.63 clips/min	72 W79°C	✓ Measured
LTX-Video (image to video)	1.41 clips/min	71 W82°C	✓ Measured
Wan 2.2 TI2V-5B (image to video)	✕ Won't fit needs ~31 GB	VRAM-gated at this precision	✓ Measured
Stable Video Diffusion XT	✕ Won't fit needs ~40 GB	VRAM-gated at this precision	✓ Measured
CogVideoX-5B I2V	✕ Won't fit needs ~44 GB	VRAM-gated at this precision	✓ Measured

Video Generation frames/s 3

Wan 2.1 1.3B0.219

CogVideoX-2B0.179

Wan 2.2 5B (720p)0.15

Workload	Result	Telemetry	Data
Wan 2.2 5B (720p)	0.15 frames/s	16.7 GB peak70 W84°C335 s/clip	✓ Measured CPU offload
CogVideoX-2B	0.18 frames/s	72 W79°C274 s/clip	✓ Measured
Wan 2.1 1.3B	0.22 frames/s	72 W78°C223.9 s/clip	✓ Measured

Image to 3D assets/hour 4

TripoSR Image-to-3D967.5

TripoSG Image-to-3D83.2

TRELLIS Image-to-3D68.8

TRELLIS.2 Image-to-3D23.6

Workload	Result	Telemetry	Data
TripoSR Image-to-3D	967.5 assets/hour	46 W71°C	✓ Measured
TripoSG Image-to-3D	83.2 assets/hour	70 W80°C	✓ Measured
TRELLIS Image-to-3D	68.8 assets/hour		✓ Measured
TRELLIS.2 Image-to-3D	23.6 assets/hour		✓ Measured

Music Generation x realtime 3

ACE-Step 1.512.84

ACE-Step v1 3.5B8.26

MusicGen Small1.01

Workload	Result	Telemetry	Data
MusicGen Small	1.01 x realtime	53 W57°C	✓ Measured
ACE-Step 1.5	12.84 x realtime	56 W76°C	✓ Measured
ACE-Step v1 3.5B	8.26 x realtime	68 W64°C	✓ Measured

Sound Effects x realtime 3

MiDashengLM-Gen0.71

EzAudio XL0.38

MOSS-SoundEffect v2.00.37

Workload	Result	Telemetry	Data
EzAudio XL	0.38 x realtime	72 W82°C	✓ Measured
MOSS-SoundEffect v2.0	0.37 x realtime	72 W79°C	✓ Measured
MiDashengLM-Gen	0.71 x realtime	71 W79°C	✓ Measured

Speech to Text x realtime 1

Workload	Result	Telemetry	Data
Whisper large-v3	70.11 x realtime	56 W52°C	✓ Measured

Text to Speech x realtime 1

Workload	Result	Telemetry	Data
Kokoro TTS 82M	97.35 x realtime	34 W46°C	✓ Measured

Depth Estimation images/min 2

Workload	Result	Telemetry	Data
Depth Anything V2 Small	916.66 images/min	28 W47°C	✓ Measured
Depth Anything V2 Large	770.37 images/min	30 W48°C	✓ Measured

Segmentation images/min 2

Workload	Result	Telemetry	Data
SAM ViT-Base	315.89 images/min	40 W52°C	✓ Measured
SAM ViT-Huge	70.46 images/min	69 W45°C	✓ Measured

Vision Language images/min 2

Workload	Result	Telemetry	Data
Florence-2 Base	174.65 images/min	38 W41°C	✓ Measured
Florence-2 Large	93.77 images/min	44 W43°C	✓ Measured

Fine-Tuning train tok/s 4

TinyLlama 1.1B LoRA4756

Qwen2.5 1.5B LoRA3720.9

SmolLM2 1.7B LoRA3240.2

Qwen2.5 7B LoRA1063.9

Workload	Result	Telemetry	Data
TinyLlama 1.1B LoRA	4756 train tok/s	71 W42°C	✓ Measured
Qwen2.5 1.5B LoRA	3720.9 train tok/s	71 W45°C	✓ Measured
SmolLM2 1.7B LoRA	3240.2 train tok/s	72 W49°C	✓ Measured
Qwen2.5 7B LoRA	1063.9 train tok/s	72 W57°C	✓ Measured

LLM Serving serve tok/s 4

TinyLlama 1.1B served2584.3

Qwen2.5 1.5B served1827

SmolLM2 1.7B served1524.9

Qwen2.5 7B served469.9

Workload	Result	Telemetry	Data
TinyLlama 1.1B served	2584.3 serve tok/s	72 W59°C	✓ Measured
Qwen2.5 1.5B served	1827 serve tok/s	72 W56°C	✓ Measured
SmolLM2 1.7B served	1524.9 serve tok/s	72 W58°C	✓ Measured
Qwen2.5 7B served	469.9 serve tok/s	72 W65°C	✓ Measured

Speculative Decoding x vs solo 1

Workload	Result	Telemetry	Data
Qwen2.5 1.5B + 0.5B draft	0.97 x vs solo		✓ Measured

How we measured this. Every result comes from our own pinned, reproducible AI suite, 12 workloads: the Qwen3-4B to Llama-70B LLM ladder (llama.cpp, Q4_K_M), SDXL / Z-Image / FLUX-dev generation, FLUX-Kontext / Qwen-Edit editing, and LTX / Wan video, run first-party on rented hardware with under 0.5% run-to-run variance. Peak VRAM, power draw, temperature and tokens-per-watt are captured per workload. “Won’t fit” rows are real data: where a model exceeds the card’s VRAM at the tested precision we record a hard gate rather than silently dropping to a smaller quant. Measured 2026-07-10 · harness 2.0.0.

NVIDIA L4 specifications

Architecture	Ada Lovelace
CUDA cores	7,424
VRAM	24GB GDDR6
Memory bus	192-bit
Memory bandwidth	300 GB/s
Boost clock	2,040 MHz
TDP	72 W
Process	TSMC 4N
Interface	PCIe 4.0 x16
Release date	2023-03-21
Launch MSRP	$2,500

Verdict, capable, but 24GB sets the ceiling

NVIDIA L4 scores 5.0/100, #41 of 102. It ran 6 of 12; 5 exceeded its 24GB. Every figure here is our own measurement.

Relative performance: where the NVIDIA L4 lands

100% = this card, AI & Machine Learning headline metric (AI Score). #20 of 21 datacenter cards in this vertical.

GPU	%	AI Score
NVIDIA A40	354%	17.7
NVIDIA A100 40GB SXM4	340%	17
NVIDIA A100 40GB PCIe	334%	16.7
NVIDIA A10G	128%	6.4
NVIDIA L4	100%	5
NVIDIA T4	58%	2.9

One tier up: NVIDIA A10G (+28%) · compare head-to-head
One tier down: NVIDIA T4 (-42%) · compare head-to-head

← All AI & Machine Learning GPU rankings

The silicon

Transistors	35,800 million
Die size	294.5 mm²
Process node	4 nm
Fabricated by	TSMC
Transistor density	121.6 million per mm²

Denser than 94% of the 746 cards we have silicon data for. Density is the clearest measure of what a process node bought: a card that gained it without growing the die got its speed from the fab rather than the architecture.

Silicon figures from Wikipedia (CC BY-SA 4.0). Benchmarks on this page are our own. Compare every chip.

What this card can build

Whole-job timings, composed from our measured per-model results on this card.

Workflow	Time	Energy	Basis
Depth pass on a batch	17 s	0.03 Wh	measured
Voiceovers from scripts	36 s	0.17 Wh	measured
Podcast episode pass	2 min	1.6 Wh	all 3 stages measured
Transcribe and subtitle videos	4.4 min	3.97 Wh	measured
Masking run	7.5 min	8.2 Wh	measured
Caption a training dataset	10.8 min	7.82 Wh	measured
Product catalogue cutout	12.1 min	13.61 Wh	all 2 stages measured
24-frame storyboard	12.2 min	13.69 Wh	all 2 stages measured
3D game asset kit	22.6 min	4.63 Wh	all 2 stages measured
Full codebase review	36.4 min	37.51 Wh	measured
Short social clips	60.3 min	69.46 Wh	all 3 stages measured
Photos to 3D models	66.2 min	0.06 Wh	all 2 stages measured

Can't run: Animate a batch of images (needs Wan 2.2 TI2V-5B (image to video)), Product shoot, start to finish (needs FLUX.1 Kontext dev), Product photo shoot (needs FLUX.1 Kontext dev), Photo restoration batch (needs FLUX.1 Kontext dev), Restore and enlarge photos (needs FLUX.1 Kontext dev), Character sheet, 12 poses (needs FLUX.1 dev), 6-panel comic page (needs FLUX.1 dev), Long-form article batch (needs Llama 3.3 70B).

Rent or buy?

This card is $2,500 to buy. The cheapest listed rate on Vast.ai is $0.312/hour, but that is the floor: we budget $0.374/hour, a 20% premium, because idle time, storage and unavailable cheap instances all land on the same bill. At that rate buying wins after 6,677 GPU-hours. Below it you are paying for idle silicon.

How you would use it	GPU-hours a year	Rental cost a year	Time to break even
2 hours a day, hobby	730	$273	9.1 years
8 hours a day, working on it	2,920	$1,093	2.3 years
24/7, always-on agent	8,760	$3,280	9.1 months

At hobby usage this card is very unlikely to pay for itself before it is superseded. Rent it. Rental figures include a 20% premium over the cheapest listed rate. Ignores electricity, resale and the fact that a rented card can be a newer one tomorrow.

Rental price

$0.440/hr+12.8% since 2026-08-14low $0.269 · high $0.440

Cheapest of the RunPod and Vast on-demand rates we see, sampled daily. Spot and interruptible pricing runs lower.