Llama 3 8B · 33 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Llama 3 8B?

We ran Llama 3 8B (Q4_K_M, llama.cpp) on 33 GPUs, from budget cards to datacenter parts, and measured generation speed, prompt processing, power draw and peak VRAM on every one. The fastest was NVIDIA H100 80GB HBM3 at 264.4 tok/s.

Benchmarked weights: bartowski/Meta-Llama-3-8B-Instruct-GGUF

Fastest we measured

NVIDIA H100 80GB HBM3

264.4 tok/s on Llama 3 8B, the ceiling. Measured on our bench. 80GB of VRAM, $30,000 at launch.

Pros

• 264.4 tok/s on Llama 3 8B
• 80GB, clears the Llama 3 8B floor
• Rentable by the hour rather than bought

Cons

• 700W board rating
• Datacenter or workstation hardware, not a retail purchase

Best consumer card

NVIDIA GeForce RTX 5090

259.2 tok/s on Llama 3 8B, fastest card you can buy at retail. Measured on our bench. 32GB of VRAM, $1,999 at launch.

Pros

• 259.2 tok/s on Llama 3 8B
• 32GB, clears the Llama 3 8B floor

Cons

• 575W board rating

Cheapest card that runs it

GeForce RTX 4060

52.35 tok/s on Llama 3 8B, lowest launch price that still fits. Measured on our bench. 8GB of VRAM, $299 at launch.

Pros

• 52.35 tok/s on Llama 3 8B
• 8GB, clears the Llama 3 8B floor

Cons

• 115W board rating

Best value

NVIDIA GeForce RTX 3060 Ti

85.46 tok/s on Llama 3 8B, most speed per dollar. Measured on our bench. 8GB of VRAM, $399 at launch. That is 214.2 tok/s per $1,000 of launch price.

Pros

• 85.46 tok/s on Llama 3 8B
• 8GB, clears the Llama 3 8B floor

Cons

• 200W board rating

What the numbers show. Across 33 GPUs measured on our own bench, H100 80GB HBM3 is fastest at 264 tok/s. Per dollar of launch price, RTX 5060 Ti gives the most (238.6 tok/s per $1,000). The fastest card with 16GB or less is RTX 5080 at 159 tok/s. The measured peak was ~5GB, so it runs on 6GB cards and up. That floor is a property of the model, so it applies to every GPU, measured or not.

About Llama 3 8B. Llama 3 8B: from meta-llama, 8.0B parameters, on Hugging Face since April 2024, Llama 3 Community licence (gated: accept the terms first). 1,091,068 downloads in the last 30 days and 1 community quantizations.

How it compares. H100 80GB HBM3: Llama 3 8B 264.4 tok/s, Llama-3.1-8B 261.8 (8B), Meta-Llama-3.1-8B 261.7 (8B), Qwen3 8B 244.2 (8B), DeepSeek-R1-0528-Qwen3-8B 245.3 (8B). Llama 3 8B beats all 4 here.

Cost on a rented GPU. 1M generated tokens of Llama 3 8B: $0.18 on a RTX 3080 ($0.082/hr, 2.2 hours), $2.24 on a H100 80GB HBM3 ($2.14/hr, 63 min, 12.4x the cost).

Speed tiers for Llama 3 8B. 30+ tok/s: 33 (RTX 5090, RTX 4090, RTX 3090 Ti). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Llama 3 8B writes anything it reads the input: 14510.4 tok/s on the RTX 5090 (0.3s for a 4,000-token prompt), 10920.2 on the RTX 4090 (0.4s), 1212.0 on the T4 (3.3s). Long documents and big code files feel this number more than the generation speed.

VRAM for Llama 3 8B. Measured peak 4.7GB, so 8GB is the smallest common card size; smallest card it ran on: RTX 3070 Ti (8GB). With long context: Q4_K_M 7GB (tested), Q2_K 5GB, Q3_K_M 6GB, Q5_K_M 9GB, Q6_K 10GB.

Power on Llama 3 8B. Most efficient: H100 80GB HBM3, 258W, 0.27 kWh per 1M generated tokens. Hungriest: RTX 5090, 297W, 0.32 kWh. At $0.15/kWh: $0.041 per 1M generated tokens.

Our verdict

Fastest on Llama 3 8B: NVIDIA H100 80GB HBM3, 264.4 tok/s. Best desktop card: NVIDIA GeForce RTX 5090, 259.2 tok/s. Cheapest consumer card that ran it: GeForce RTX 4060 ($299, 52.35 tok/s). Cheapest to rent per job: NVIDIA GeForce RTX 3080, $0.18 per 1M generated tokens.

FAQ

What GPU do I need to run Llama 3 8B?

About 5GB. Cheapest consumer card that ran it: GeForce RTX 4060 (8GB, 52.35 tok/s).

How fast is Llama 3 8B on the NVIDIA GeForce RTX 5090?

259.2 tok/s at 297W, 98% of the NVIDIA H100 80GB HBM3.

How much does it cost to run Llama 3 8B in the cloud?

$0.18 per 1M generated tokens on a NVIDIA GeForce RTX 3080 at $0.082/hr, cheapest of 28 rentable cards we measured.

Can I run Llama 3 8B on a 12GB, 16GB or 24GB card?

It used 4.7GB at the precision we tested. 12GB: yes; 16GB: yes; 24GB: yes.

Is the RTX 4090 or the RTX 3090 faster for Llama 3 8B?

The RTX 4090: 167.7 vs 146.3 tok/s, 15% faster on our bench.

Should I buy or rent a RTX 5090 for Llama 3 8B?

Its $1,999 launch price buys 5,139 rented hours at $0.39/hr, enough for about 4,795x 1M generated tokens of Llama 3 8B. Buy only if you'll run more than that.

How we test

llama.cpp llama-bench at Q4_K_M, 512-token prompt and 128 generated tokens, three runs after a warmup, full GPU offload, with power and VRAM sampled throughout. Token generation is memory-bandwidth-bound, so the ranking tracks bandwidth closely, which makes it a fair guide to cards we haven't run yet.