Dolphin 3.0 R1 Mistral 24B · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Dolphin 3.0 R1 Mistral 24B?

Dolphin 3.0 R1 Mistral 24B is the most interesting model in the Dolphin line: an uncensored tune with R1-style reasoning on a Mistral 24B base, a judge that thinks before it criticizes. Measured on 11 GPUs (llama.cpp, Q4_K_M): 121 tok/s on the B300, ~15GB peak VRAM.

Benchmarked weights: bartowski/cognitivecomputations_Dolphin3.0-R1-Mistral-24B-GGUF

Fastest we measured

NVIDIA B300

121.0 tok/s on Dolphin 3.0 R1 Mistral 24B, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 121.0 tok/s on Dolphin 3.0 R1 Mistral 24B
• 288GB, clears the Dolphin 3.0 R1 Mistral 24B floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Cheapest card that runs it

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

93.62 tok/s on Dolphin 3.0 R1 Mistral 24B, lowest launch price that still fits. Measured on our bench. 96GB of VRAM, $8,565 at launch.

Pros

• 93.62 tok/s on Dolphin 3.0 R1 Mistral 24B
• 96GB, clears the Dolphin 3.0 R1 Mistral 24B floor
• Rentable by the hour rather than bought

Cons

• 600W board rating
• Datacenter or workstation hardware, not a retail purchase

A reasoning judge is the consensus endgame. Our consensus recipe needs two things at the referee seat: honesty (uncensored, so critiques aren't softened) and rigor (reasoning, so verdicts come with a visible chain of logic). This model is the only one we benchmark that has both. Put your Qwen and DeepSeek answers in front of it and it produces a reasoned, unhedged adjudication, the closest thing to a working code-review-bot personality in the open-source lineup. Vintage note: as a recent-generation Dolphin it inherits the line's gradual drift toward neutrality, the early Dolphins were blunter. For the judge role that's an acceptable trade; the reasoning trace matters more than maximum edge.

Fit and speed. The ~15GB floor means a 16GB card technically holds it, but tightly, long reasoning traces plus context will crowd the ceiling, so 24GB is the comfortable home. Speed lands mid-pack for its size: 121 tok/s peak, 109 on the H200 at 257W. Remember it pays the reasoning-token tax on top: on the A10G's 31 tok/s, a long adjudication takes a minute. If the judge is in your daily loop, feed it bandwidth.

How it compares. H100 80GB HBM3: Dolphin 3.0 R1 Mistral 24B 107.8 tok/s, Devstral Small 24B 107.8, Dolphin Mistral 24B Venice 109.1, Dolphin-Mistral-24B-Venice-Edition 105.6, Mistral Small 24B 105.6. 1 of 4 beat Dolphin 3.0 R1 Mistral 24B here.

Cost on a rented GPU. 1M generated tokens of Dolphin 3.0 R1 Mistral 24B: $2.10 on a A100 40GB SXM4 ($0.47/hr, 4.5 hours), $15.94 on a B300 ($6.94/hr, 2.3 hours, 7.6x the cost).

Speed tiers for Dolphin 3.0 R1 Mistral 24B. 30+ tok/s: 9 (B300, B200, H200); 10-30 tok/s: 2 (L4, T4). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Dolphin 3.0 R1 Mistral 24B writes anything it reads the input: 4902.0 tok/s on the RTX PRO 6000 Blackwell Workstation Edition (0.8s for a 4,000-token prompt), 419.8 on the T4 (9.5s). Long documents and big code files feel this number more than the generation speed.

VRAM for Dolphin 3.0 R1 Mistral 24B. Measured peak 13.9GB, so 16GB is the smallest common card size; smallest card it ran on: T4 (16GB). With long context: Q4_K_M 16GB (tested), Q2_K 10GB, Q3_K_M 13GB, Q5_K_M 19GB, Q6_K 22GB.

Power on Dolphin 3.0 R1 Mistral 24B. Most efficient: RTX PRO 6000 Blackwell Workstation Edition, 199W, 0.59 kWh per 1M generated tokens. Hungriest: B300, 348W, 0.80 kWh. At $0.15/kWh: $0.089 per 1M generated tokens.

Our verdict

Dolphin 3.0 R1 Mistral 24B: 121 tok/s peak, ~15GB floor: uncensored and reasoning-trained, which makes it our pick for the judge seat in a serious consensus setup. Give it 24GB for comfort and enough tok/s that its deliberation doesn't become your bottleneck.

FAQ

What makes this the best consensus judge?

It's the only model we benchmark that is both uncensored (no diplomatic softening of critiques) and R1-reasoning-trained (verdicts arrive with an explicit chain of logic). Referee work needs both properties.

Can a 16GB card run it?

Just. Measured peak was ~15GB at Q4_K_M, so 16GB fits with minimal headroom. For long contexts plus reasoning traces, a 24GB card is the realistic recommendation.

How fast is it in practice?

121 tok/s peak, 109 on an H200. Budget for the thinking tokens: a thorough adjudication can run to a thousand trace tokens, so at A10G speed (31 tok/s) expect closer to a minute per verdict.

Dolphin R1 24B or the Venice edition?

Identical hardware profile (both ~15GB, both ~121 tok/s peak). This one reasons before answering; Venice answers directly. Judge seat: this one. Fast uncensored chat: Venice.

What panel do you pair it with?

Qwen3 32B (or the 30B MoE) plus DeepSeek-R1 32B as the answering pair, this as referee. All three fit a 24GB card sequentially, a complete consensus stack on one consumer GPU.

Does the reasoning trace slow down its verdicts a lot?

Measurably: a thorough adjudication burns hundreds of thinking tokens before the verdict appears. At the H200's 109 tok/s that's a few seconds; at the A10G's 31 tok/s it's approaching a minute. If verdicts are in your interactive loop, prioritize tok/s when picking its card.