Qwen3 8B · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for Qwen3 8B?

Qwen3 8B is what we'd call the base tier for a real chat experience, the class of model the cheap API endpoints actually serve. We measured it on 11 GPUs with llama.cpp at Q4_K_M: 261 tok/s on the B300 at the top, and a ~6GB measured VRAM floor that puts it within reach of nearly every desktop card sold in the last five years.

Benchmarked weights: Qwen/Qwen3-8B-GGUF

My take: this is the quality baseline. Below roughly this tier, local LLM use is pipeline work; at 8B it becomes something you'd actually let users talk to. If you're deciding what the minimum viable model for a product experience is, this is where I'd draw the line, 4B if you're squeezed, 8B if you want answers that consistently hold up. It's also the sweet spot for a first local setup: the ~6GB floor means an 8GB card runs it with room for context, and you don't need to think about quantization tricks or offloading.

What the numbers show. The B300 tops the chart at 261 tok/s but burns 275W doing it, 0.95 tok/W. The H200 lands 5% slower at 248 tok/s while drawing 127W, which is 1.95 tok/W and the best efficiency on the board. That's a pattern you'll see across our small-model results: Blackwell wins the headline, Hopper wins the power bill. At the affordable end, an Intel Arc A580 at $179 clears the VRAM floor, and even the ancient T4 still generates 37 tok/s, faster than most people read.

Our verdict

Qwen3 8B at Q4_K_M: 261 tok/s on the B300, ~6GB floor, and the H200 doing 95% of the B300's speed at less than half the power. Our position: this is the base tier for user-facing chat, the model class the budget APIs serve, and the natural first model for anyone with an 8GB card.

FAQ

Can an 8GB card run Qwen3 8B?

Yes: we measured ~6GB peak at Q4_K_M, leaving headroom for context on an 8GB card. The $179 Intel Arc A580 clears the floor; so does any RTX 3050 or better.

Is Qwen3 8B good enough for a real product?

In our view it's the baseline where user-facing chat stops feeling like a toy. Below it (1.7B/4B) you're in pipeline-component territory; above it (14B/30B-A3B) you're buying quality with VRAM.

What's the most efficient GPU for Qwen3 8B?

The H200: 248 tok/s at 127W measured, 1.95 tok/W, double the efficiency of the B300 that beats it by only 5%. For sustained serving, that difference is your power bill.

Qwen3 8B or Qwen3 14B?

8B needs ~6GB and does 261 tok/s peak in our tests; 14B needs ~10GB and tops out at 165 tok/s. If your card has 12GB+, the 14B's quality is worth the speed loss; on an 8GB card the 8B is the honest fit.

How does it compare to Llama 3.1 8B?

Same weight class and similar throughput characteristics. Both are bandwidth-bound at Q4_K_M. Qwen3 has the newer training recipe; we benchmark both so you can compare tok/s per card directly on their pages.