Gemma 3 27B · 10 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Gemma 3 27B?

Gemma 3 27B is Google's open flagship: and the model we keep comparing against Mistral Small 24B, because they compete for the same slot in a local stack. Our measurements: 93 tok/s peak on the B300, ~18GB VRAM floor, on the same pinned llama.cpp harness as everything else here.

Benchmarked weights: bartowski/google_gemma-3-27b-it-GGUF

Fastest we measured

NVIDIA B300

93.44 tok/s on Gemma 3 27B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 93.44 tok/s on Gemma 3 27B
• 288GB, clears the Gemma 3 27B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Cheapest card that runs it

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

72.1 tok/s on Gemma 3 27B, lowest launch price that still fits. Measured on our bench. 96GB of VRAM, $8,565 at launch.

Pros

• 72.1 tok/s on Gemma 3 27B
• 96GB, clears the Gemma 3 27B floor
• Rentable by the hour rather than bought

Cons

• 600W board rating
• Datacenter or workstation hardware, not a retail purchase

The honest head-to-head. Here's our experience, numbers attached: Mistral Small 24B measured 28% faster (119 vs 93 tok/s peak) with a lighter floor (~15GB vs ~18GB), and in day-to-day use we haven't seen output quality from the 27B that overturns that hardware math. Gemma's case rests on what Mistral doesn't have, vision support and stronger multilingual range, plus Google's instruction tuning, which some people prefer. But if your work is English text and code, our advice is to benchmark both against your own prompts before committing the extra VRAM; ours kept landing on the Mistral.

Fit notes. The ~18GB floor is awkward: past every 16GB card, tight on 20GB, comfortable only at 24GB: where it suddenly competes with the dense 32B class (DeepSeek, Qwen, QwQ at ~21GB) and the far faster Qwen3 30B-A3B MoE (309 tok/s). That's the strategic problem this model faces in our lineup: the cards that fit it also fit stronger or faster options. The H100's 85 tok/s at 216W (0.39 tok/W) led efficiency in our runs.

How it compares. H100 80GB HBM3: Gemma 3 27B 84.77 tok/s, Mistral Small 24B (Q3_K_M) 84.0, Codestral 22B (Q3_K_M) 87.21, Qwen3.8 27B 80.5 (28B), Qwen3.6 27B 79.74 (28B). 1 of 4 beat Gemma 3 27B here.

Cost on a rented GPU. 1M generated tokens of Gemma 3 27B: $2.76 on a A100 40GB SXM4 ($0.47/hr, 5.9 hours), $20.63 on a B300 ($6.94/hr, 3.0 hours, 7.5x the cost).

Speed tiers for Gemma 3 27B. 30+ tok/s: 8 (B300, B200, H100 80GB HBM3); 10-30 tok/s: 2 (A10G, L4). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Gemma 3 27B writes anything it reads the input: 4074.5 tok/s on the RTX PRO 6000 Blackwell Workstation Edition (1.0s for a 4,000-token prompt), 842.7 on the L4 (4.7s). Long documents and big code files feel this number more than the generation speed.

VRAM for Gemma 3 27B. Measured peak 16.9GB, so 24GB is the smallest common card size; smallest card it ran on: A10G (24GB). With long context: Q4_K_M 18GB (tested), Q2_K 12GB, Q3_K_M 15GB, Q5_K_M 21GB, Q6_K 25GB.

Power on Gemma 3 27B. Most efficient: H100 80GB HBM3, 216W, 0.71 kWh per 1M generated tokens. Hungriest: B200, 380W, 1.23 kWh. At $0.15/kWh: $0.11 per 1M generated tokens.

Our verdict

Gemma 3 27B: 93 tok/s peak, ~18GB floor: a solid open flagship that loses its head-to-head with Mistral Small 24B on speed and VRAM in our data, and faces the whole 32B class at 24GB. Its vision and multilingual strengths are real; make sure they're what you're paying the VRAM for.

FAQ

Gemma 3 27B or Mistral Small 24B?

Our measurements: Mistral is 28% faster (119 vs 93 tok/s) and needs less VRAM (~15 vs ~18GB), and our task results favored it too. Gemma wins on vision and multilingual work, if you need those, it's the one; otherwise test both on your prompts.

What GPU does it need?

Realistically 24GB, the ~18GB measured floor rules out 16GB cards and leaves 20GB ones tight. At 24GB, also consider the 32B-class models and the Qwen3 30B MoE that fit the same card.

What is it actually best at?

Multimodal and multilingual breadth at flagship-open scale. It reads images, handles many languages gracefully, and carries Google's instruction polish, a genuinely distinct profile even where raw benchmarks don't lead.

How fast is it on rented hardware?

93 tok/s on the B300, 85 on an H100 at the class-best 0.39 tok/W. At the budget end the A10G's 26 tok/s is patient-single-user territory.

Is Gemma 4 worth waiting for instead?

Gemma 4 12B is already in our database (156 tok/s, ~9GB) and shows the line's newest recipe in the mid size. No 27B-class Gemma 4 has crossed our bench yet; when one does, it gets the same treatment.