Dolphin Mistral 24B Venice · 10 GPUs measured first-party · llama.cpp Q4_K_M · Updated July 2026

What GPU Do You Need for Dolphin Mistral 24B Venice?

Dolphin Mistral 24B Venice Edition is the direct-answer counterpart to the R1 Dolphin: the same uncensored Mistral 24B foundation, tuned for straight responses instead of reasoning traces. Measured on 10 GPUs (llama.cpp, Q4_K_M): 121 tok/s on the B300, ~15GB peak, and a standout efficiency run on the H200: 109 tok/s at just 110W.

Benchmarked weights: dphn/Dolphin-Mistral-24B-Venice-Edition-GGUF

The fast half of the uncensored 24B pair. Venice skips the thinking-token tax: every generated token is answer, which makes it the better daily chat model of the two Mistral Dolphins, same brain size, no deliberation overhead. It's also become a privacy-community favorite (the Venice name comes from the private-AI platform that commissioned it), and running it locally is the logical conclusion of that: an unaligned, capable 24B that never leaves your machine. Set expectations accordingly: today's Dolphins, Venice included, are more neutral than the early generation was. The line has softened over the years even while staying well clear of aligned-model filtering. For most private local use that middle ground is exactly right.

The efficiency headline. The H200 row deserves attention: 109 tok/s at 110.0W measured, 0.99 tok/W, nearly triple the efficiency of the B300 that beats it by 11%. That's the widest Hopper-vs-Blackwell efficiency gap in our 24B data. On owned hardware, the ~15GB floor makes 16GB cards a snug fit and 24GB cards comfortable; at 33 tok/s even an A10G delivers usable chat.

Our verdict

Dolphin Mistral 24B Venice: 121 tok/s peak, ~15GB floor, no reasoning overhead, the fast, private, uncensored daily driver of the Dolphin line. If the R1 Dolphin is your judge, this is your workhorse; together they're the whole uncensored mid-size stack.

FAQ

What is the Venice Edition?

An uncensored Dolphin tune of Mistral 24B commissioned by the Venice private-AI platform. Locally it means one thing: a capable direct-answer model with no alignment shaping, running entirely on your hardware.

What GPU does it need?

~15GB measured peak at Q4_K_M, a 16GB card fits it tightly, 24GB comfortably. Peak measured speed was 121 tok/s (B300); the H200's 109 tok/s at 110W is the efficiency pick.

Venice or the R1 Dolphin 24B?

Same hardware, different temperament. Venice answers immediately, better for chat and drafting. The R1 tune reasons first, better for judging and hard problems. Most Dolphin users we'd advise to run Venice daily and call R1 for verdicts.

Is it fast enough on consumer cards?

The 24B class lands around a third of 8B speeds on equal hardware, but stays interactive: expect chat-comfortable rates on anything with real bandwidth, and 33 tok/s even on an A10G.

Why run an uncensored model at all?

Because alignment tuning doesn't just refuse, it shapes and hedges everything. For private local use, review work, and creative writing, an unshaped model doing exactly what you asked is the point.

What's the cheapest sensible card to own for it?

A 16GB card technically clears the ~15GB floor, but with almost no context headroom. The practical answer is a used RTX 3090: 24GB swallows the model plus long contexts, and this direct-answer 24B doesn't pay the reasoning-token tax, so even older bandwidth stays interactive.