What GPU does Gemma 3 12B need?

A 12GB card, comfortably. Measured peak was ~9GB at Q4_K_M, the roomiest fit of any 12-14B model we've benchmarked. Arc B580 ($179) or RTX 3060 12GB both work with context headroom.

Gemma 3 12B or Qwen3 14B / Phi-4 14B?

Measured speeds are close (161 / 165 / 177 tok/s peaks). Gemma wins on vision support and VRAM headroom; Phi on raw speed and polish; Qwen on ecosystem. None dominates, match the temperament to your task.

Can it look at images?

Yes, like the 4B and 27B, Gemma 3 12B is multimodal. If your workflow mixes text tasks with reading screenshots or photos, it's the only 12GB-class option in our lineup that does both.

Is 12B enough for serious local work?

It's the threshold, our working rule is 12-14B minimum for dependable output, with the 27-32B range as the sweet spot. This model sits exactly at that entry line.

What's the best value card for it?

The Arc B580 at $179 is the cheapest comfortable host. On the rental side, the H200's 153 tok/s at 151W was the efficiency standout in our runs.

Gemma 3 12B · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for Gemma 3 12B?

Gemma 3 12B is the practical middle of Google's open lineup, the size where our '12-14B minimum for serious local work' rule starts being satisfied. Measured on 11 GPUs (llama.cpp, Q4_K_M): 161 tok/s on the B300, ~9GB peak VRAM, so 12GB cards host it with room to spare.

Benchmarked weights: bartowski/google_gemma-3-12b-it-GGUF

161.27tok/s

Fastest: NVIDIA B300

measured, 3-run llama-bench

~9GB

VRAM needed (measured peak)

GPU-independent, applies to every card

GPUs measured

same pinned harness

1.01tok/W

Most efficient: NVIDIA H200

real power sampling, not TDP

What GPU Do You Need for Gemma 3 12B?, tok/s, fastest 11

NVIDIA B300

161.27 tok/s

NVIDIA H200

153.26 tok/s

NVIDIA B200

151.41 tok/s

NVIDIA H100 80GB HBM3

150.58 tok/s

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

138.09 tok/s

NVIDIA A100 80GB SXM4

91.45 tok/s

NVIDIA A100 40GB SXM4

89.94 tok/s

NVIDIA L40S

80.38 tok/s

NVIDIA A10G

55.67 tok/s

NVIDIA L4

30.99 tok/s

NVIDIA T4

21.85 tok/s

Measured on our own bench. A card absent from this chart has not been run on this model yet, or cannot fit it.

Gemma 3 12B. Measured generation speed by GPU

NVIDIA B300161.27

NVIDIA H200153.26

NVIDIA B200151.41

NVIDIA H100 80GB HBM3150.58

NVIDIA RTX PRO 6000 Blackwell Workstation Edition138.09

NVIDIA A100 80GB SXM491.45

NVIDIA A100 40GB SXM489.94

NVIDIA L40S80.38

NVIDIA A10G55.67

NVIDIA L430.99

NVIDIA T421.85

GPU	tok/s	Prompt t/s	tok/W	Avg power
NVIDIA B300	161.27	3323	0.51	315.0 W
NVIDIA H200	153.26	5382	1.01	151.2 W
NVIDIA B200	151.41	5768.4	0.52	290.7 W
NVIDIA H100 80GB HBM3	150.58	5432.3	0.64	233.5 W
NVIDIA RTX PRO 6000 Blackwell Workstation Edition	138.09	7537.1	0.73	188.5 W
NVIDIA A100 80GB SXM4	91.45	2845.7	0.57	159.3 W
NVIDIA A100 40GB SXM4	89.94	2855.6	0.83	109.0 W
NVIDIA L40S	80.38	5895.3	0.47	172.2 W
NVIDIA A10G	55.67	2483.6	0.44	125.6 W
NVIDIA L4	30.99	1797	0.55	56.5 W
NVIDIA T4	21.85	768.2	0.38	58.2 W

A capable middle-of-the-pack citizen. The family verdict applies at 12B too: Gemma is good at everything and the best at nothing. Against its direct rivals in our database, Qwen3 14B (165 tok/s, ~10GB), Phi-4 14B (177 tok/s, ~10GB), DeepSeek-R1 14B (156 tok/s, reasoning), it trades blows without winning a category outright. What it does bring: vision support (the others are text-only), strong multilingual output, and the lightest floor of the group at ~9GB. If your 12GB card needs one do-everything model that can also read screenshots, that combination is a legitimate reason to choose it.

The measurements. 161 tok/s peak, and a notably strong H200 row: 153 tok/s at 151W (1.01 tok/W). At the budget end the L4 manages 31 tok/s, single-user viable, no more. The ~9GB floor is the practical headline: it's the roomiest fit of any 12-14B-class model we benchmark, leaving real context headroom on 12GB cards where its rivals run tight.

Our verdict

Gemma 3 12B: 161 tok/s peak and the lightest floor in its class (~9GB), the generalist option at the size where local LLM work starts being real. Choose it for vision + multilingual breadth on a 12GB card; choose its rivals when you need their specializations.

FAQ

What GPU does Gemma 3 12B need?: A 12GB card, comfortably. Measured peak was ~9GB at Q4_K_M, the roomiest fit of any 12-14B model we've benchmarked. Arc B580 ($179) or RTX 3060 12GB both work with context headroom.
Gemma 3 12B or Qwen3 14B / Phi-4 14B?: Measured speeds are close (161 / 165 / 177 tok/s peaks). Gemma wins on vision support and VRAM headroom; Phi on raw speed and polish; Qwen on ecosystem. None dominates, match the temperament to your task.
Can it look at images?: Yes, like the 4B and 27B, Gemma 3 12B is multimodal. If your workflow mixes text tasks with reading screenshots or photos, it's the only 12GB-class option in our lineup that does both.
Is 12B enough for serious local work?: It's the threshold, our working rule is 12-14B minimum for dependable output, with the 27-32B range as the sweet spot. This model sits exactly at that entry line.
What's the best value card for it?: The Arc B580 at $179 is the cheapest comfortable host. On the rental side, the H200's 153 tok/s at 151W was the efficiency standout in our runs.