Qwen3 8B · 57 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Qwen3 8B?

Qwen3 8B is what we'd call the base tier for a real chat experience, the class of model the cheap API endpoints actually serve. We measured it on 57 GPUs with llama.cpp at Q4_K_M: 261 tok/s on the B300 at the top, and a ~6GB measured VRAM floor that puts it within reach of nearly every desktop card sold in the last five years.

Benchmarked weights: Qwen/Qwen3-8B-GGUF

Fastest we measured

NVIDIA B300

261.4 tok/s on Qwen3 8B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 261.4 tok/s on Qwen3 8B
• 288GB, clears the Qwen3 8B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Best consumer card

NVIDIA GeForce RTX 5090

243.9 tok/s on Qwen3 8B, fastest card you can buy at retail. Measured on our bench. 32GB of VRAM, $1,999 at launch.

Pros

• 243.9 tok/s on Qwen3 8B
• 32GB, clears the Qwen3 8B floor

Cons

• 575W board rating

Cheapest card that runs it

NVIDIA GeForce GTX 1660

33.74 tok/s on Qwen3 8B, lowest launch price that still fits. Measured on our bench. 6GB of VRAM, $219 at launch.

Pros

• 33.74 tok/s on Qwen3 8B
• 6GB, clears the Qwen3 8B floor

Cons

• 120W board rating

Best value

NVIDIA GeForce RTX 5060

77.26 tok/s on Qwen3 8B, most speed per dollar. Measured on our bench. 8GB of VRAM, $249 at launch. That is 310.3 tok/s per $1,000 of launch price.

Pros

• 77.26 tok/s on Qwen3 8B
• 8GB, clears the Qwen3 8B floor

Cons

• 145W board rating

My take: this is the quality baseline. Below roughly this tier, local LLM use is pipeline work; at 8B it becomes something you'd actually let users talk to. If you're deciding what the minimum viable model for a product experience is, this is where I'd draw the line, 4B if you're squeezed, 8B if you want answers that consistently hold up. It's also the sweet spot for a first local setup: the ~6GB floor means an 8GB card runs it with room for context, and you don't need to think about quantization tricks or offloading.

What the numbers show. The B300 tops the chart at 261 tok/s but burns 275W doing it, 0.95 tok/W. The H200 lands 5% slower at 248 tok/s while drawing 127W, which is 1.95 tok/W and the best efficiency on the board. That's a pattern you'll see across our small-model results: Blackwell wins the headline, Hopper wins the power bill. At the affordable end, an Intel Arc A580 at $179 clears the VRAM floor, and even the ancient T4 still generates 37 tok/s, faster than most people read.

About Qwen3 8B. Qwen3 8B: from Qwen, 8.2B parameters, on Hugging Face since April 2025, Apache 2.0 licence. 14,968,230 downloads in the last 30 days and 6 community quantizations.

How it compares. H100 80GB HBM3: Qwen3 8B 244.2 tok/s, DeepSeek-R1-0528-Qwen3-8B 245.3 (8B), Qwen2.5-VL 7B Instruct 267.7 (8B), Llama-3.1-8B 261.8 (8B), Llama 3 8B 264.4 (8B). All 4 beat Qwen3 8B here.

Cost on a rented GPU. 1M generated tokens of Qwen3 8B: $0.16 on a RTX 3060 ($0.036/hr, 4.4 hours), $7.37 on a B300 ($6.94/hr, 64 min, 47.1x the cost).

Speed tiers for Qwen3 8B. 30+ tok/s: 57 (RTX 5090, RTX 4090, RTX 3090 Ti). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Qwen3 8B writes anything it reads the input: 13393.1 tok/s on the RTX 5090 (0.3s for a 4,000-token prompt), 11239.7 on the RTX 4090 (0.4s), 144.9 on the GTX 1660 Super (27.6s). Long documents and big code files feel this number more than the generation speed.

VRAM for Qwen3 8B. Measured peak 4.7GB, so 8GB is the smallest common card size; smallest card it ran on: GTX 1660 Ti (6GB). With long context: Q4_K_M 6GB (tested), Q2_K 4GB, Q3_K_M 5GB, Q5_K_M 7GB, Q6_K 9GB.

Power on Qwen3 8B. Most efficient: H100 80GB HBM3, 174W, 0.20 kWh per 1M generated tokens. Hungriest: RTX 3090 Ti, 291W, 0.52 kWh. At $0.15/kWh: $0.030 per 1M generated tokens.

Our verdict

Qwen3 8B at Q4_K_M: 261 tok/s on the B300, ~6GB floor, and the H200 doing 95% of the B300's speed at less than half the power. Our position: this is the base tier for user-facing chat, the model class the budget APIs serve, and the natural first model for anyone with an 8GB card.

FAQ

Can an 8GB card run Qwen3 8B?

Yes: we measured ~6GB peak at Q4_K_M, leaving headroom for context on an 8GB card. The $179 Intel Arc A580 clears the floor; so does any RTX 3050 or better.

Is Qwen3 8B good enough for a real product?

In our view it's the baseline where user-facing chat stops feeling like a toy. Below it (1.7B/4B) you're in pipeline-component territory; above it (14B/30B-A3B) you're buying quality with VRAM.

What's the most efficient GPU for Qwen3 8B?

The H200: 248 tok/s at 127W measured, 1.95 tok/W, double the efficiency of the B300 that beats it by only 5%. For sustained serving, that difference is your power bill.

Qwen3 8B or Qwen3 14B?

8B needs ~6GB and does 261 tok/s peak in our tests; 14B needs ~10GB and tops out at 165 tok/s. If your card has 12GB+, the 14B's quality is worth the speed loss; on an 8GB card the 8B is the honest fit.

How does it compare to Llama 3.1 8B?

Same weight class and similar throughput characteristics. Both are bandwidth-bound at Q4_K_M. Qwen3 has the newer training recipe; we benchmark both so you can compare tok/s per card directly on their pages.