Llama 3.2 1B · 24 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Llama 3.2 1B?

Llama 3.2 1B holds the single fastest LLM record in our entire database: 893 tokens per second on the RTX PRO 6000 Blackwell. Measured on 24 GPUs (llama.cpp, Q4_K_M) with a ~2GB floor, it's Meta's smallest model, and the benchmark ceiling for what 'fast' means in local inference.

Benchmarked weights: bartowski/Llama-3.2-1B-Instruct-GGUF

Fastest we measured

NVIDIA GeForce RTX 5090

1060.1 tok/s on Llama 3.2 1B, the ceiling. Measured on our bench. 32GB of VRAM, $1,999 at launch.

Pros

• 1060.1 tok/s on Llama 3.2 1B
• 32GB, clears the Llama 3.2 1B floor

Cons

• 575W board rating

Best consumer card

NVIDIA GeForce RTX 4090

752.5 tok/s on Llama 3.2 1B, fastest card you can buy at retail. Measured on our bench. 24GB of VRAM, $1,599 at launch.

Pros

• 752.5 tok/s on Llama 3.2 1B
• 24GB, clears the Llama 3.2 1B floor

Cons

• 450W board rating

Cheapest card that runs it

NVIDIA GeForce GTX 1660 Super

193.4 tok/s on Llama 3.2 1B, lowest launch price that still fits. Measured on our bench. 6GB of VRAM, $229 at launch.

Pros

• 193.4 tok/s on Llama 3.2 1B
• 6GB, clears the Llama 3.2 1B floor

Cons

• 125W board rating

Best value

NVIDIA GeForce RTX 5060

377.0 tok/s on Llama 3.2 1B, most speed per dollar. Measured on our bench. 8GB of VRAM, $249 at launch. That is 1513.9 tok/s per $1,000 of launch price.

Pros

• 377.0 tok/s on Llama 3.2 1B
• 8GB, clears the Llama 3.2 1B floor

Cons

• 145W board rating

A good base, with one caveat. Llama 3.2 1B is exactly what it says: a clean, well-trained base model with the largest tooling ecosystem in open source behind it. As raw material for fine-tunes and as maximum-velocity pipeline glue, it's excellent. Our honest ranking for the tiny tier, though: if the job is being a small *assistant*, Phi-4 Mini (3.8B) answers noticeably better for a modest speed cost; if the job is running client-side on user devices, Qwen3 0.6B's sub-1B size keeps weak hardware viable. This model's lane is speed itself, batch work where 893 tok/s is the entire specification.

What the record run tells us. The top three cards are separated by barely 1%: 893, 889, 882 tok/s. At 1B parameters, nothing can differentiate big silicon, the model simply can't load it. Which makes the sensible deployment the opposite of glamorous: an L4 does 258 tok/s at 46W measured, a seven-year-old T4 does 201, and both are overkill for most pipelines. This is the model class where hardware stops mattering.

About Llama 3.2 1B. Llama 3.2 1B: from meta-llama, 1.2B parameters, on Hugging Face since September 2024, Llama 3.2 Community licence (gated: accept the terms first). 9,073,830 downloads in the last 30 days and 3 community quantizations.

How it compares. H100 80GB HBM3: Llama 3.2 1B 880.6 tok/s, gemma-3-1b 504.2 (1B), Qwen2.5-1.5B 537.2 (2B), Qwen2.5-Coder-1.5B 538.4 (2B), Qwen2-1.5B 536.5 (2B). Llama 3.2 1B beats all 4 here.

Cost on a rented GPU. 1M generated tokens of Llama 3.2 1B: $0.033 on a RTX 3060 ($0.036/hr, 55 min), $0.10 on a RTX 5090 ($0.39/hr, 16 min, 3.1x the cost).

Speed tiers for Llama 3.2 1B. 30+ tok/s: 24 (RTX 5090, RTX 4090, RTX 5080). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Llama 3.2 1B writes anything it reads the input: 53758.8 tok/s on the RTX 5090 (0.1s for a 4,000-token prompt), 45428.9 on the RTX 4090 (0.1s), 998.4 on the GTX 1660 Super (4.0s). Long documents and big code files feel this number more than the generation speed.

VRAM for Llama 3.2 1B. Measured peak 0.9GB, so 8GB is the smallest common card size; smallest card it ran on: GTX 1660 Super (6GB). With long context: Q4_K_M 1GB (tested), Q2_K 2GB, Q3_K_M 2GB, Q5_K_M 2GB, Q6_K 2GB.

Power on Llama 3.2 1B. Most efficient: RTX 5070 Ti, 78W, 31.2 Wh per 1M generated tokens. Hungriest: B200, 294W, 92.6 Wh.

Our verdict

Llama 3.2 1B: 893 tok/s, the fastest LLM result we've ever measured, with a ~2GB floor that runs on anything. A good base model and the definitive speed play; for tiny-tier assistant quality, our pick shifts to Phi-4 Mini.

FAQ

What's the fastest GPU result for Llama 3.2 1B?

893 tok/s on the RTX PRO 6000 Blackwell, the fastest LLM measurement in our whole database. The B300 and B200 land within 1%: at 1B parameters, top-end hardware is indistinguishable.

What hardware does it actually need?

Almost none, ~2GB peak at Q4_K_M. A T4 from 2018 delivers 201 tok/s. For batch pipelines, the cheapest efficient card you can find (L4: 258 tok/s at 46W) is the right answer.

Llama 3.2 1B or Phi-4 Mini as a small assistant?

Phi-4 Mini, in our view, its 3.8B answers are clearly stronger, and it still measured 398 tok/s peak. Choose this 1B when velocity or fine-tuning is the goal, not conversation quality.

Can it run on user devices like Qwen3 0.6B?

It's borderline, at ~1.2B parameters it's past the sub-1B line where weak devices keep up. On decent laptops yes; for the broadest 'runs on anything the user owns' guarantee, the 0.6B remains the safer bet.

What is it best at?

Being a base. The Llama ecosystem, fine-tuning recipes, tooling, deployment paths, is the deepest in open source, and this is its lightest expression. Speculative decoding drafts, edge summarizers, and custom tunes are its natural jobs.