Phi-4 Mini 3.8B · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Phi-4 Mini 3.8B?

Phi-4 Mini is Microsoft's 3.8B distillation of the Phi-4 recipe, and our pick for the best small assistant in the database. Measured on 11 GPUs (llama.cpp, Q4_K_M): 398 tok/s on the B300, ~4GB peak VRAM, with the RTX PRO 6000 delivering the best efficiency at 2.27 tok/W.

Benchmarked weights: bartowski/microsoft_Phi-4-mini-instruct-GGUF

Fastest we measured

NVIDIA B300

398.2 tok/s on Phi-4 Mini 3.8B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 398.2 tok/s on Phi-4 Mini 3.8B
• 288GB, clears the Phi-4 Mini 3.8B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Cheapest card that runs it

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

391.0 tok/s on Phi-4 Mini 3.8B, lowest launch price that still fits. Measured on our bench. 96GB of VRAM, $8,565 at launch.

Pros

• 391.0 tok/s on Phi-4 Mini 3.8B
• 96GB, clears the Phi-4 Mini 3.8B floor
• Rentable by the hour rather than bought

Cons

• 600W board rating
• Datacenter or workstation hardware, not a retail purchase

The small-tier quality pick. In the under-4B class, this is the model we'd actually hand to users. Microsoft's synthetic-textbook training gives it answer quality that punches above its parameter count, in our experience clearly ahead of Llama 3.2's small pair for assistant work, which is why our tiny-tier advice reads: Qwen3 0.6B for client-side deployment, Phi-4 Mini for the best small brain. At 398 tok/s peak it gives up only ~10% speed to the Llama 3B while being the noticeably better conversationalist.

Deployment math. The ~4GB floor is the sweet spot for modest hardware: 6GB laptop GPUs and every desktop card from the last half-decade clear it with room. The chart's flat top (398/392/391 across three card classes) says the usual: small models can't use big silicon, so deploy on efficiency: the PRO 6000's 391 tok/s at 172W is elegant, but an L4 at 87 tok/s and 55W is the volume-serving bargain.

How it compares. H100 80GB HBM3: Phi-4 Mini 3.8B 388.9 tok/s, gemma-2-2b 392.3 (3B), gemma-2-2b-it-abliterated 392.6, Qwen2.5-Coder-3B 394.0 (3B), Qwen2.5-3B 395.5 (3B). All 4 beat Phi-4 Mini 3.8B here.

Cost on a rented GPU. 1M generated tokens of Phi-4 Mini 3.8B: $0.55 on a A100 40GB SXM4 ($0.47/hr, 70 min), $4.84 on a B300 ($6.94/hr, 42 min, 8.7x the cost).

Speed tiers for Phi-4 Mini 3.8B. 30+ tok/s: 11 (B300, H200, RTX PRO 6000 Blackwell Workstation Edition). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Phi-4 Mini 3.8B writes anything it reads the input: 20459.3 tok/s on the RTX PRO 6000 Blackwell Workstation Edition (0.2s for a 4,000-token prompt), 2352.1 on the T4 (1.7s). Long documents and big code files feel this number more than the generation speed.

VRAM for Phi-4 Mini 3.8B. Measured peak 3.1GB, so 8GB is the smallest common card size; smallest card it ran on: T4 (16GB). With long context: Q4_K_M 3GB (tested), Q2_K 3GB, Q3_K_M 3GB, Q5_K_M 4GB, Q6_K 4GB.

Power on Phi-4 Mini 3.8B. Most efficient: RTX PRO 6000 Blackwell Workstation Edition, 172W, 0.12 kWh per 1M generated tokens. Hungriest: B200, 329W, 0.24 kWh. At $0.15/kWh: $0.018 per 1M generated tokens.

Our verdict

Phi-4 Mini: 398 tok/s peak, ~4GB floor, and the best answers-per-parameter in our small-model lineup, our quality pick for the tiny tier. Fits nearly anything, including modest laptop GPUs; pair it with a big-model escalation path and most everyday queries never need the big model.

FAQ

Why pick Phi-4 Mini over Llama 3.2 3B?

Answer quality. The Phi training recipe (curated synthetic data) yields unusually strong reasoning for 3.8B, in our use it's the clear conversationalist of the small tier. Llama 3.2 wins on fine-tuning ecosystem and a ~10% speed edge instead.

What GPU does it need?

~4GB measured peak at Q4_K_M, 6GB cards run it comfortably, including laptop GPUs. Peak speed was 398 tok/s (B300), but even a T4 manages 66 tok/s.

Is it good enough as a primary assistant?

For everyday queries, drafting, explaining, quick answers, genuinely yes. Route hard reasoning and long-context work to a 14B+ model and this handles the volume tier admirably.

Phi-4 Mini or Qwen3 4B?

Same weight class, both excellent. Our split: Phi for answer polish, Qwen3 4B for multilingual breadth and the broader Qwen ecosystem. Both need ~4GB; try each against your actual prompts.

What about Phi-4 14B?

Bigger brother, 177 tok/s peak, ~10GB floor. Solid, but at 14B it competes with DeepSeek's distill and the 24B coding models, where our preference usually goes to those. Mini competes at 4B, where it's the standout.