Llama 3.1 8B · 71 cards measured first-party · Updated October 2026

How Fast Does Llama 3.1 8B Run on Each GPU?

Llama 3.1 8B is our reference LLM, the one workload nearly every GPU in our fleet can run, which makes it the cleanest bandwidth comparison we have. It's also the model most people actually use day to day, because at ~8GB it fits almost anywhere and it's fast enough to feel like a real assistant.

Benchmarked weights: unsloth/Llama-3.1-8B-Instruct-GGUF

Fastest we measured

NVIDIA B300

287.2 tok/s on Llama 3.1 8B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 287.2 tok/s on Llama 3.1 8B
• 288GB, clears the Llama 3.1 8B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Best consumer card

NVIDIA GeForce RTX 5090

268.1 tok/s on Llama 3.1 8B, fastest card you can buy at retail. Measured on our bench. 32GB of VRAM, $1,999 at launch.

Pros

• 268.1 tok/s on Llama 3.1 8B
• 32GB, clears the Llama 3.1 8B floor

Cons

• 575W board rating

Cheapest card that runs it

Intel Arc B580

60.3 tok/s on Llama 3.1 8B, lowest launch price that still fits. Anchored estimate. 12GB of VRAM, $179 at launch.

Pros

• 60.3 tok/s on Llama 3.1 8B
• 12GB, clears the Llama 3.1 8B floor

Cons

• 190W board rating

Best value

NVIDIA GeForce RTX 5060

79.05 tok/s on Llama 3.1 8B, most speed per dollar. Measured on our bench. 8GB of VRAM, $249 at launch. That is 317.5 tok/s per $1,000 of launch price.

Pros

• 79.05 tok/s on Llama 3.1 8B
• 8GB, clears the Llama 3.1 8B floor

Cons

• 145W board rating

This is the purest bandwidth test on the site. At 8B the model fits everywhere, nothing is gated, and nothing is compute-bound, so the leaderboard below is close to a direct ranking of memory bandwidth across 100+ GPUs. If you want to understand why VRAM speed matters more than tensor cores for local LLMs, this is the chart that shows it.

Because this workload is bandwidth-bound, the ranking above tracks memory bandwidth far more closely than core counts or price. A card with fewer tensor cores and faster memory will beat a card with the opposite.

That's the reason we run twelve workloads instead of publishing one score. A GPU isn't fast or slow. It's fast at some things and gated out of others, and which of those matters depends entirely on what you're actually going to run.

Our verdict

NVIDIA B300 tops our Llama 3.1 8B leaderboard at 287.2 tok/s (measured), 755% of the way clear of the slowest card that still fits. This is a bandwidth workload: buy memory speed, not tensor cores.

FAQ

What is the fastest GPU for Llama 3.1 8B?

NVIDIA B300, at 287.2 tok/s on our bench, a first-party measurement. It carries 288GB of VRAM. Of the 102 cards we have Llama 3.1 8B data for, 100 can run it at all.

How much VRAM do I need for Llama 3.1 8B?

~8GB at Q4_K_M. Almost everything clears it, which is exactly what makes it a good yardstick.

Why does the Llama 3.1 8B ranking look different from your other benchmarks?

Because this workload is bandwidth-bound, the ranking above tracks memory bandwidth far more closely than core counts or price. A card with fewer tensor cores and faster memory will beat a card with the opposite. That's why we publish twelve separate workloads rather than one blended score, the ordering genuinely changes depending on the job.

Are these Llama 3.1 8B numbers measured or estimated?

Both, and every row says which. 71 of the 102 cards here were rented and run by us on the same harness. The remainder are anchored estimates interpolated per workload against those measurements. We never blend the two silently, if a row says Estimated, we have not run that card.

Can I rent a GPU to run Llama 3.1 8B instead of buying one?

Yes, and for the cards at the top of this leaderboard it's the only realistic option, most of them have no retail channel at all. It's also how we got these numbers: we rented the hardware by the hour rather than buying it. That's worth considering before you spend on a card to find out whether it's fast enough.

Why publish cards that can't run Llama 3.1 8B?

Nearly every card in our fleet clears this workload's VRAM floor, so there are no gates on this page. That's unusual, most of our suite excludes a meaningful chunk of the market.

How we test

Every ranking on this page comes from our own benchmark runs, not vendor claims. Cards marked Measured were rented and run by us; cards marked Estimated are interpolated per workload against those measured anchors and are labelled on every row, we never blend the two silently. LLMs run on llama.cpp (llama-bench) at Q4_K_M with -p 512 -n 128. Diffusion and video run on diffusers/ComfyUI at BF16, with SDXL at FP16. Each workload gets a warmup pass plus multiple timed runs (5 for small LLMs, 3 for large models and images, 2 for video); we publish the mean as the result and the minimum as the 1% low. Run-to-run variance is under 0.5%. Telemetry, power, temperature, utilisation, clocks, peak VRAM, is sampled at 1 Hz for the duration of every run. Where a model exceeds a card's VRAM we publish a hard won't-fit result rather than quietly dropping to a smaller quantisation. A card that can't run a model scores zero on it. Silently swapping precision to make a number appear would make every number on this site meaningless. All figures are single-GPU, single-stream, batch-size-1. That is the honest way to measure what one card does for one user, and it is deliberately not how a datacenter serves a model. Vendor and MLPerf figures use large batches across many GPUs and will be far higher. Neither is wrong, they answer different questions. Ours answers 'what will this card do for me'.