Why does speed matter more for QwQ than other models?

It generates long thinking traces before every answer, often thousands of tokens. At 83 tok/s that's tolerable; on a slow card it's minutes per question. Judge cards for QwQ by the tok/s column above everything else.

What GPU fits QwQ 32B?

24GB cards and up. Measured peak was ~21GB at Q4_K_M. RX 7900 XTX, RTX 3090, RTX 4090, or rented datacenter cards (B300 topped our chart at 83 tok/s).

Is the reasoning actually worth the token cost?

On hard problems, math, logic, subtle bugs, genuinely yes: the trace catches mistakes direct-answer models commit to. On easy questions it's pure overhead. Route accordingly: QwQ for the hard 10%, a fast model for the rest.

QwQ 32B or Qwen3 32B?

Identical hardware requirements and near-identical measured speed (83 vs 84 tok/s at the top). QwQ trades latency for deliberation; Qwen3 32B answers directly. Many setups keep both and route by difficulty.

What's the best way to use QwQ locally?

In a consensus rotation: same prompt to QwQ, a coder model, and a non-Qwen 32B, then compare. One 24GB card hosts all three (one at a time), and the disagreement between them is where the value shows up.

QwQ 32B · 10 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for QwQ 32B?

QwQ 32B is Qwen's reasoning model: it thinks out loud before answering, which multiplies token output per question and makes generation speed matter more than for any normal model. We measured it on 10 GPUs (llama.cpp, Q4_K_M): 83 tok/s on the B300, ~21GB peak VRAM, same memory class as the dense coder.

Benchmarked weights: Qwen/QwQ-32B-GGUF

82.91tok/s

Fastest: NVIDIA B300

measured, 3-run llama-bench

~21GB

VRAM needed (measured peak)

GPU-independent, applies to every card

GPUs measured

same pinned harness

0.36tok/W

Most efficient: NVIDIA H100 80GB HBM3

real power sampling, not TDP

What GPU Do You Need for QwQ 32B?, tok/s, fastest 10

NVIDIA B300

82.91 tok/s

NVIDIA B200

78.47 tok/s

NVIDIA H100 80GB HBM3

75.82 tok/s

NVIDIA H200

75.77 tok/s

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

66.07 tok/s

NVIDIA A100 40GB SXM4

43.51 tok/s

NVIDIA A100 80GB SXM4

43.27 tok/s

NVIDIA L40S

34.46 tok/s

NVIDIA A10G

23.52 tok/s

NVIDIA L4

11.72 tok/s

Measured on our own bench. A card absent from this chart has not been run on this model yet, or cannot fit it.

QwQ 32B. Measured generation speed by GPU

NVIDIA B30082.91

NVIDIA B20078.47

NVIDIA H100 80GB HBM375.82

NVIDIA H20075.77

NVIDIA RTX PRO 6000 Blackwell Workstation Edition66.07

NVIDIA A100 40GB SXM443.51

NVIDIA A100 80GB SXM443.27

NVIDIA L40S34.46

NVIDIA A10G23.52

NVIDIA L411.72

GPU	tok/s	Prompt t/s	tok/W	Avg power
NVIDIA B300	82.91	1351.6	0.25	327.8 W
NVIDIA B200	78.47	2576.7	0.22	348.9 W
NVIDIA H100 80GB HBM3	75.82	2291.2	0.36	212.7 W
NVIDIA H200	75.77	2309.1	0.34	224.6 W
NVIDIA RTX PRO 6000 Blackwell Workstation Edition	66.07	3431.8	0.3	218.3 W
NVIDIA A100 40GB SXM4	43.51	1172	0.28	152.8 W
NVIDIA A100 80GB SXM4	43.27	1179.6	0.27	159.5 W
NVIDIA L40S	34.46	2380.4	0.2	168.5 W
NVIDIA A10G	23.52	1032	0.16	148.9 W
NVIDIA L4	11.72	667.2	0.2	59.5 W

Reasoning changes the economics. A reasoning model doesn't just answer, it burns hundreds or thousands of 'thinking' tokens first. At 83 tok/s peak, a hard question with a long trace takes real wall-clock time, and on a 23 tok/s card it takes minutes. So the usual 'anything above 30 tok/s feels fine' rule doesn't apply here: with QwQ, throughput is the user experience. Plan for that when choosing hardware. This is the one 32B where the speed column should drive your decision.

Why I still run it. The reasoning does make a difference: on math, logic and tricky debugging, the thinking trace catches errors a direct-answer model confidently commits. My favorite use is in a consensus setup: same prompt to QwQ, a dense coder, and a model from outside the Qwen family, then reconcile. The disagreements are where the insight is. All of them fit a 24GB card one at a time (~21GB floor here), so a single used 3090 or a 7900 XTX can host the whole rotation, no datacenter rental required for single-stream Q4 work.

Our verdict

QwQ 32B: 83 tok/s peak, ~21GB floor, and a token-hungry reasoning style that makes speed the whole ballgame. Worth it when the problem is genuinely hard, and at its best as one voice in a multi-model consensus rather than your only local brain.

FAQ

Why does speed matter more for QwQ than other models?: It generates long thinking traces before every answer, often thousands of tokens. At 83 tok/s that's tolerable; on a slow card it's minutes per question. Judge cards for QwQ by the tok/s column above everything else.
What GPU fits QwQ 32B?: 24GB cards and up. Measured peak was ~21GB at Q4_K_M. RX 7900 XTX, RTX 3090, RTX 4090, or rented datacenter cards (B300 topped our chart at 83 tok/s).
Is the reasoning actually worth the token cost?: On hard problems, math, logic, subtle bugs, genuinely yes: the trace catches mistakes direct-answer models commit to. On easy questions it's pure overhead. Route accordingly: QwQ for the hard 10%, a fast model for the rest.
QwQ 32B or Qwen3 32B?: Identical hardware requirements and near-identical measured speed (83 vs 84 tok/s at the top). QwQ trades latency for deliberation; Qwen3 32B answers directly. Many setups keep both and route by difficulty.
What's the best way to use QwQ locally?: In a consensus rotation: same prompt to QwQ, a coder model, and a non-Qwen 32B, then compare. One 24GB card hosts all three (one at a time), and the disagreement between them is where the value shows up.