What GPU do I need for Codestral 22B?

16GB. Measured peak was ~14GB at Q4_K_M. That's its quiet superpower: it's the strongest completion model in our database that fits 16GB cards like the RTX 4060 Ti 16GB.

What is fill-in-the-middle and why does it matter?

FIM lets the model complete with code both above and below the cursor. Which is how editors actually work. Chat models fake it; Codestral was trained for it, and completion quality shows the difference.

Codestral or Devstral?

Both. They're different halves. Codestral is the inline-completion specialist; Devstral is tuned for agentic workflows (multi-step tasks, tool use). A serious local coding setup runs Codestral in the editor and Devstral in the agent loop.

Is 120 tok/s fast enough for completions?

Comfortably. Completion UX needs the first tokens fast, and Codestral's high prompt-ingestion speed (up to 3,258 t/s measured) means big-file context loads quickly. Even mid-tier cards deliver instant-feeling suggestions.

Codestral 22B or Qwen2.5-Coder 32B?

Codestral for 16GB cards and completion-first use. The Qwen needs 24GB and answers more deeply in chat. If you have 24GB, the strongest stack skips both dense options for Qwen3 Coder 30B-A3B, and keeps Codestral for FIM.

Codestral 22B · 10 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for Codestral 22B?

Codestral 22B is Mistral's dedicated code model, trained for fill-in-the-middle completion across 80+ languages. We measured it on 10 GPUs (llama.cpp, Q4_K_M): 120 tok/s on the B300, ~14GB peak VRAM, which lands it squarely on 16GB cards.

Benchmarked weights: bartowski/Codestral-22B-v0.1-GGUF

119.81tok/s

Fastest: NVIDIA B300

measured, 3-run llama-bench

~14GB

VRAM needed (measured peak)

GPU-independent, applies to every card

GPUs measured

same pinned harness

0.65tok/W

Most efficient: NVIDIA H200

real power sampling, not TDP

What GPU Do You Need for Codestral 22B?, tok/s, fastest 10

NVIDIA B300

119.81 tok/s

NVIDIA B200

112.16 tok/s

NVIDIA H200

111.34 tok/s

NVIDIA H100 80GB HBM3

110.05 tok/s

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

92.7 tok/s

NVIDIA A100 40GB SXM4

64.17 tok/s

NVIDIA A100 80GB SXM4

63.26 tok/s

NVIDIA L40S

50.08 tok/s

NVIDIA A10G

32.53 tok/s

NVIDIA L4

18.16 tok/s

Measured on our own bench. A card absent from this chart has not been run on this model yet, or cannot fit it.

Codestral 22B. Measured generation speed by GPU

NVIDIA B300119.81

NVIDIA B200112.16

NVIDIA H200111.34

NVIDIA H100 80GB HBM3110.05

NVIDIA RTX PRO 6000 Blackwell Workstation Edition92.7

NVIDIA A100 40GB SXM464.17

NVIDIA A100 80GB SXM463.26

NVIDIA L40S50.08

NVIDIA A10G32.53

NVIDIA L418.16

GPU	tok/s	Prompt t/s	tok/W	Avg power
NVIDIA B300	119.81	1929.6	0.36	328.4 W
NVIDIA B200	112.16	3257.8	0.31	365.7 W
NVIDIA H200	111.34	3137.3	0.65	170.5 W
NVIDIA H100 80GB HBM3	110.05	3185.8	0.49	224.3 W
NVIDIA RTX PRO 6000 Blackwell Workstation Edition	92.7	4798.2	0.62	150.1 W
NVIDIA A100 40GB SXM4	64.17	1599.8	0.42	151.9 W
NVIDIA A100 80GB SXM4	63.26	1638.9	0.36	174.4 W
NVIDIA L40S	50.08	3526.7	0.28	180.2 W
NVIDIA A10G	32.53	1103.1	0.31	105.1 W
NVIDIA L4	18.16	969.4	0.33	55.8 W

The completion specialist in the stack. Our model philosophy is specialists over generalists, and Codestral is a clean example: it exists to complete code: fill-in-the-middle is its native trick, which matters because that's how real editors request completions (cursor in the middle of a file, context on both sides). Slot it as the IDE-facing model in a coding stack: Codestral serving completions, with a bigger agent model (Devstral, or Qwen's Coder MoE) handling chat, refactors and multi-file work.

Fit and speed notes. The ~14GB floor is the story for owners: this is the strongest completion model that fits a 16GB card, RTX 4060 Ti 16GB, RX 7600 XT territory, where the 20GB+ models can't follow. Speed lands at 120 tok/s peak and 111 on the H200 at 171W; for inline completion, anything above ~30 tok/s feels instant, which even the A10G's 33 tok/s clears. One number to appreciate: 1,930-3,260 prompt t/s at the top means big file contexts load fast. Completion latency is dominated by prompt ingestion, and this model ingests quickly.

Our verdict

Codestral 22B: 120 tok/s peak, ~14GB floor, the best code-completion specialist that fits a 16GB card. Pair it with an agent-class model and you've split the coding workload the way the models were actually trained: Codestral completes, the big model thinks.

FAQ

What GPU do I need for Codestral 22B?: 16GB. Measured peak was ~14GB at Q4_K_M. That's its quiet superpower: it's the strongest completion model in our database that fits 16GB cards like the RTX 4060 Ti 16GB.
What is fill-in-the-middle and why does it matter?: FIM lets the model complete with code both above and below the cursor. Which is how editors actually work. Chat models fake it; Codestral was trained for it, and completion quality shows the difference.
Codestral or Devstral?: Both. They're different halves. Codestral is the inline-completion specialist; Devstral is tuned for agentic workflows (multi-step tasks, tool use). A serious local coding setup runs Codestral in the editor and Devstral in the agent loop.
Is 120 tok/s fast enough for completions?: Comfortably. Completion UX needs the first tokens fast, and Codestral's high prompt-ingestion speed (up to 3,258 t/s measured) means big-file context loads quickly. Even mid-tier cards deliver instant-feeling suggestions.
Codestral 22B or Qwen2.5-Coder 32B?: Codestral for 16GB cards and completion-first use. The Qwen needs 24GB and answers more deeply in chat. If you have 24GB, the strongest stack skips both dense options for Qwen3 Coder 30B-A3B, and keeps Codestral for FIM.