Qwen2.5-Coder 32B · 10 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for Qwen2.5-Coder 32B?

Qwen2.5-Coder 32B was the local coding flagship of its generation: a dense 32B tuned for code, needing ~21GB at Q4_K_M and topping our chart at 83 tok/s on the B300. We measured it on 10 GPUs with the same pinned llama.cpp harness as every other model on this site.

Benchmarked weights: bartowski/Qwen2.5-Coder-32B-Instruct-GGUF

Where a dense 32B coder fits now. Two ways to use this model well. First, as the strong single model on a 24GB card: 24GB clears the ~21GB floor, and 30-40 tok/s-class generation is comfortable for chat-style coding. Second, and more interesting: as one voice in a consensus setup. Running the same prompt through two or three different 32B-class models, this, QwQ, something outside the Qwen family, and reconciling the answers gets you meaningfully better results than any single model. That's my current view of the best local strategy: dense 32B models as ensemble members, not soloists. Since the Qwen3 Coder MoE arrived, the MoE is the daily driver and this is the second opinion.

A value note the chart hides. The B300 tops the board at 83 tok/s, but dense 32B generation is bandwidth-bound and single-stream, so datacenter silicon is mostly wasted on it. On the sister dense model (Qwen3 32B) we measured an RTX 5090 at 71 tok/s against the B300's 84: a consumer card delivering 85% of the flagship datacenter chip's output on this workload. Even a used RTX 3090 holds 38 tok/s. Renting big iron for one-at-a-time Q4 inference is the least efficient way to spend AI money, big cards earn their rate on batch serving, not solo chat.

Our verdict

Qwen2.5-Coder 32B: 83 tok/s peak, ~21GB floor, a dense coding brain that any 24GB card can hold. Our advice: run it as the strong second opinion in a consensus setup alongside the faster Qwen3 Coder MoE, and don't rent datacenter silicon for single-stream Q4 inference, a 24GB consumer card does half the B300's speed at a sliver of the cost.

FAQ

What GPU do I need for Qwen2.5-Coder 32B?

24GB minimum. Measured peak was ~21GB at Q4_K_M. RX 7900 XTX, RTX 3090/3090 Ti, RTX 4090. Nothing smaller fits it at this quant.

Is it worth renting a datacenter GPU for?

For single-user inference, no. Dense 32B generation is bandwidth-bound: on the sister Qwen3 32B, an RTX 5090 measured 71 tok/s against the B300's 84, 85% of the output from a card you can own. Datacenter cards earn their rates on batch serving, not solo chat.

Qwen2.5-Coder 32B or Qwen3 Coder 30B-A3B?

The MoE for daily use, nearly 4× the generation speed (318 vs 83 tok/s at each chart's top) in the same VRAM class. Keep the dense 32B as a second opinion on hard problems: different architecture, different failure modes.

What's a consensus setup?

Running the same prompt through several strong models, say this, QwQ 32B, and a non-Qwen 32B, and reconciling the answers. In our experience it beats any single local model, and all three fit a 24GB card one at a time.

How fast is it on a consumer card?

We measured its sister dense model (Qwen3 32B) at 71 tok/s on an RTX 5090, 44 on a 4090 and 38 on a 3090. Expect this model to land in the same bands. The 5090 makes dense 32B genuinely comfortable; older 24GB cards are chat-speed but slow for agent loops.