Dolphin 3.0 Llama 3.1 8B · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Dolphin 3.0 Llama 3.1 8B?

Dolphin 3.0 Llama 3.1 8B puts the uncensored Dolphin treatment on Meta's Llama 3.1 base, an everyday-size assistant that answers without alignment hedging. Measured on 11 GPUs (llama.cpp, Q4_K_M): 286 tok/s on the B300, ~6GB peak VRAM, 2.49 tok/W best-case efficiency.

Benchmarked weights: dphn/Dolphin3.0-Llama3.1-8B-GGUF

Fastest we measured

NVIDIA B300

285.9 tok/s on Dolphin 3.0 Llama 3.1 8B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 285.9 tok/s on Dolphin 3.0 Llama 3.1 8B
• 288GB, clears the Dolphin 3.0 Llama 3.1 8B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Cheapest card that runs it

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

239.3 tok/s on Dolphin 3.0 Llama 3.1 8B, lowest launch price that still fits. Measured on our bench. 96GB of VRAM, $8,565 at launch.

Pros

• 239.3 tok/s on Dolphin 3.0 Llama 3.1 8B
• 96GB, clears the Dolphin 3.0 Llama 3.1 8B floor
• Rentable by the hour rather than bought

Cons

• 600W board rating
• Datacenter or workstation hardware, not a retail purchase

The daily-driver uncensored 8B. Where we slot Dolphin X1 as a specialist judge, this one is the generalist: a Llama 3.1 foundation, the most widely-tuned base in open source, with the censorship sanded off. It's the model to reach for when you want ordinary assistant work (drafting, summarizing, Q&A) from something that won't lecture you or quietly reshape its answers. And in a review role, same argument as the whole Dolphin line: judges should be blunt, and alignment training makes models diplomatic. Worth knowing before you commit: in our experience the newer Dolphin generations, this 3.0 included, are more neutral than the early releases were. The original Dolphins on older bases had a rawer edge that the modern tunes have smoothed out. Still clearly uncensored by aligned-model standards, but the gap has narrowed over time.

Bench behavior. 286 tok/s peak and ~6GB measured put it dead level with every other 8B we run, the uncensored tune is behaviorally different, not computationally different. The efficiency line worth noting is the L4: 50 tok/s at 48.8W, comfortably interactive for a single user on a card that sips power. This tier is the cheapest 'real assistant' hardware there is.

How it compares. H100 80GB HBM3: Dolphin 3.0 Llama 3.1 8B 266.2 tok/s, Dolphin X1 8B 266.0, Qwen2.5-VL 7B Instruct 267.7 (8B), DeepSeek-R1 Distill 7B 264.4, Llama 3 8B 264.4 (8B). 1 of 4 beat Dolphin 3.0 Llama 3.1 8B here.

Cost on a rented GPU. 1M generated tokens of Dolphin 3.0 Llama 3.1 8B: $0.84 on a A100 40GB SXM4 ($0.47/hr, 106 min), $6.74 on a B300 ($6.94/hr, 58 min, 8.1x the cost).

Speed tiers for Dolphin 3.0 Llama 3.1 8B. 30+ tok/s: 11 (B300, B200, H200). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Dolphin 3.0 Llama 3.1 8B writes anything it reads the input: 12768.2 tok/s on the RTX PRO 6000 Blackwell Workstation Edition (0.3s for a 4,000-token prompt), 1174.4 on the T4 (3.4s). Long documents and big code files feel this number more than the generation speed.

VRAM for Dolphin 3.0 Llama 3.1 8B. Measured peak 5.2GB, so 8GB is the smallest common card size; smallest card it ran on: T4 (16GB). With long context: Q4_K_M 6GB (tested), Q2_K 4GB, Q3_K_M 5GB, Q5_K_M 7GB, Q6_K 9GB.

Power on Dolphin 3.0 Llama 3.1 8B. Most efficient: RTX PRO 6000 Blackwell Workstation Edition, 179W, 0.21 kWh per 1M generated tokens. Hungriest: B200, 363W, 0.37 kWh. At $0.15/kWh: $0.031 per 1M generated tokens.

Our verdict

Dolphin 3.0 Llama 3.1 8B: 286 tok/s peak, ~6GB floor, a no-hedging generalist on the most familiar base in open source. Run it as your everyday uncensored assistant on any 8GB card, or as the blunt second reviewer in a consensus stack.

FAQ

What's the difference between this and stock Llama 3.1 8B?

The Dolphin fine-tune removes alignment-driven refusals and hedging. Hardware behavior is identical in our tests. The difference is that answers are direct and critiques uncushioned.

What GPU do I need?

8GB. Measured peak was ~6GB at Q4_K_M. From a $179 Arc A580 to a rented H200 (268 tok/s at 224W), the whole range runs it well.

Is it good as a consensus judge?

Yes. That's the Dolphin family's best trick. Uncensored models review other models' answers without diplomatic softening. This one's Llama base also adds lineage diversity if your panel is Qwen-heavy.

How fast is it on budget hardware?

50 tok/s on an L4 at under 50W, 36 tok/s on a T4, both comfortably faster than reading speed. For an 8B assistant, nearly any modern GPU is enough.

Dolphin 3.0 or the 24B Dolphins?

This for speed and cheap hardware; the Mistral-based 24Bs when you want the judge itself to be smarter (they need ~15GB but run near 121 tok/s peak). Same philosophy, bigger brain, one weight class up.

What quantization and settings did you test?

Q4_K_M via llama.cpp llama-bench, 5-run pp512+tg128 protocol with logged power and VRAM, the identical pinned harness as every LLM on this site, so its 286 tok/s peak compares one-to-one against the other 8Bs in our database.