Dolphin Mistral 24B Venice · 11 GPUs measured first-party · llama.cpp Q4_K_M · Updated October 2026

What GPU Do You Need for Dolphin Mistral 24B Venice?

Dolphin Mistral 24B Venice Edition is the direct-answer counterpart to the R1 Dolphin: the same uncensored Mistral 24B foundation, tuned for straight responses instead of reasoning traces. Measured on 11 GPUs (llama.cpp, Q4_K_M): 121 tok/s on the B300, ~15GB peak, and a standout efficiency run on the H200: 109 tok/s at just 110W.

Benchmarked weights: dphn/Dolphin-Mistral-24B-Venice-Edition-GGUF

Fastest we measured

NVIDIA B300

120.9 tok/s on Dolphin Mistral 24B Venice, the ceiling. Measured on our bench. 288GB of VRAM, $40,000 at launch.

Pros

• 120.9 tok/s on Dolphin Mistral 24B Venice
• 288GB, clears the Dolphin Mistral 24B Venice floor
• Rentable by the hour rather than bought

Cons

• 1400W board rating
• Datacenter or workstation hardware, not a retail purchase

Cheapest card that runs it

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

93.67 tok/s on Dolphin Mistral 24B Venice, lowest launch price that still fits. Measured on our bench. 96GB of VRAM, $8,565 at launch.

Pros

• 93.67 tok/s on Dolphin Mistral 24B Venice
• 96GB, clears the Dolphin Mistral 24B Venice floor
• Rentable by the hour rather than bought

Cons

• 600W board rating
• Datacenter or workstation hardware, not a retail purchase

The fast half of the uncensored 24B pair. Venice skips the thinking-token tax: every generated token is answer, which makes it the better daily chat model of the two Mistral Dolphins, same brain size, no deliberation overhead. It's also become a privacy-community favorite (the Venice name comes from the private-AI platform that commissioned it), and running it locally is the logical conclusion of that: an unaligned, capable 24B that never leaves your machine. Set expectations accordingly: today's Dolphins, Venice included, are more neutral than the early generation was. The line has softened over the years even while staying well clear of aligned-model filtering. For most private local use that middle ground is exactly right.

The efficiency headline. The H200 row deserves attention: 109 tok/s at 110.0W measured, 0.99 tok/W, nearly triple the efficiency of the B300 that beats it by 11%. That's the widest Hopper-vs-Blackwell efficiency gap in our 24B data. On owned hardware, the ~15GB floor makes 16GB cards a snug fit and 24GB cards comfortable; at 33 tok/s even an A10G delivers usable chat.

How it compares. H100 80GB HBM3: Dolphin Mistral 24B Venice 109.1 tok/s, Codestral 22B 110.0, Dolphin 3.0 R1 Mistral 24B 107.8, Devstral Small 24B 107.8, Dolphin-Mistral-24B-Venice-Edition 105.6. 1 of 4 beat Dolphin Mistral 24B Venice here.

Cost on a rented GPU. 1M generated tokens of Dolphin Mistral 24B Venice: $2.10 on a A100 40GB SXM4 ($0.47/hr, 4.4 hours), $15.94 on a B300 ($6.94/hr, 2.3 hours, 7.6x the cost).

Speed tiers for Dolphin Mistral 24B Venice. 30+ tok/s: 9 (B300, B200, H200); 10-30 tok/s: 2 (L4, T4). 30 tok/s is roughly where replies outpace reading.

Reading your prompt. Before Dolphin Mistral 24B Venice writes anything it reads the input: 4927.8 tok/s on the RTX PRO 6000 Blackwell Workstation Edition (0.8s for a 4,000-token prompt), 419.8 on the T4 (9.5s). Long documents and big code files feel this number more than the generation speed.

VRAM for Dolphin Mistral 24B Venice. Measured peak 13.9GB, so 16GB is the smallest common card size; smallest card it ran on: T4 (16GB). With long context: Q4_K_M 16GB (tested), Q2_K 10GB, Q3_K_M 13GB, Q5_K_M 19GB, Q6_K 22GB.

Power on Dolphin Mistral 24B Venice. Most efficient: RTX PRO 6000 Blackwell Workstation Edition, 157W, 0.47 kWh per 1M generated tokens. Hungriest: B200, 365W, 0.89 kWh. At $0.15/kWh: $0.070 per 1M generated tokens.

Our verdict

Dolphin Mistral 24B Venice: 121 tok/s peak, ~15GB floor, no reasoning overhead, the fast, private, uncensored daily driver of the Dolphin line. If the R1 Dolphin is your judge, this is your workhorse; together they're the whole uncensored mid-size stack.

FAQ

What is the Venice Edition?

An uncensored Dolphin tune of Mistral 24B commissioned by the Venice private-AI platform. Locally it means one thing: a capable direct-answer model with no alignment shaping, running entirely on your hardware.

What GPU does it need?

~15GB measured peak at Q4_K_M, a 16GB card fits it tightly, 24GB comfortably. Peak measured speed was 121 tok/s (B300); the H200's 109 tok/s at 110W is the efficiency pick.

Venice or the R1 Dolphin 24B?

Same hardware, different temperament. Venice answers immediately, better for chat and drafting. The R1 tune reasons first, better for judging and hard problems. Most Dolphin users we'd advise to run Venice daily and call R1 for verdicts.

Is it fast enough on consumer cards?

The 24B class lands around a third of 8B speeds on equal hardware, but stays interactive: expect chat-comfortable rates on anything with real bandwidth, and 33 tok/s even on an A10G.

Why run an uncensored model at all?

Because alignment tuning doesn't just refuse, it shapes and hedges everything. For private local use, review work, and creative writing, an unshaped model doing exactly what you asked is the point.

What's the cheapest sensible card to own for it?

A 16GB card technically clears the ~15GB floor, but with almost no context headroom. The practical answer is a used RTX 3090: 24GB swallows the model plus long contexts, and this direct-answer 24B doesn't pay the reasoning-token tax, so even older bandwidth stays interactive.