The story behind the data · first person · Updated July 2026
Every AI benchmark on this site, all 51 measured cards, from a 6GB RTX 2060 to a 288GB B300, was run on hardware I never owned. I rented all of it, across Vast.ai, RunPod and Modal, and ran the exact same pinned 12-workload suite on every card. This is the story of how that works, why it's actually the right way to benchmark in 2026, and what it taught me that a spec sheet never would.
Why renting is the only honest way to do this. No reviewer owns an H200, a B300, and fifty consumer cards at once. The sites that claim coverage that wide are usually aggregating other people's numbers from other people's machines, different CPUs, drivers, settings, background load. Renting flips that: I can put the exact same pinned harness, the same model versions, the same measurement code on a 2060 and a B300 in the same week. Fewer variables than any physical fleet I could afford, and every number is mine. The harness itself is fussy on purpose: llama.cpp at a pinned commit for the LLM ladder (Qwen3 4B up to Llama 3.3 70B, Q4_K_M), diffusers/ComfyUI for SDXL, Z-Image and FLUX, plus editing and video generation. Power and temperature sampled at 1Hz through every run. Five runs per workload; if run-to-run variance goes past about half a percent, the run doesn't count.
The workflow, card by card. Find the card (Vast.ai for consumer silicon, the widest selection; RunPod for most everything else; Modal when the model download alone is a project, because a CPU container can pull the weights before the GPU meter starts). Spin it up, verify the silicon is actually what it says it is, pin the environment, run the suite, pull the logs, kill the instance. The meter discipline matters: a B300 at $7/hour rewards a plan and punishes wandering. And when a model didn't fit, and on consumer cards, most of the big ones don't, that became data too. Every 'won't fit' on this site is a recorded result with the VRAM requirement that killed it, not a gap where a number should be. A model that exceeds VRAM doesn't run slowly. It doesn't run. Buyers deserve to know that before they spend.
What surprised me. The RTX 3090, the beast, as usual. A 2020 card holding 144.9 tok/s on Llama 3.1 8B because token generation is bandwidth-bound and its 936 GB/s still holds up. The architecture shows its age on diffusion, but for pure LLM work it embarrassed cards that cost twice as much. The other surprise ran the opposite direction: the 5090 gets shockingly close to the B300 on text generation. The B300's extra muscle is real, but it shows up on image and video, the VRAM-heavy, tensor-heavy work, not on chat. That single observation reshaped how I think about the whole product stack: past a point, more VRAM is for pixels, not tokens. And the quiet lesson of the whole campaign: the gaming rig I used for the gaming suite was a machine I owned, a Ryzen 7 7800X3D build, while the AI fleet was rented by the hour. The rented fleet is still there whenever I want it. That asymmetry is the entire thesis of this site's rent-first advice.
What it cost, roughly, and why that's the point. Run the math from the public prices and you'll see the trick: even the most expensive card on the list rents for about what two coffees cost per hour, and a mid-range card rents for cents. Benchmarking 51 GPUs this way cost less than buying a single flagship card, and produced a dataset no single owned machine could ever produce. That's not a benchmarking hack. That's what compute becoming a utility looks like, and it's exactly the same math I walk through in our rent-vs-buy guides.
The whole campaign, condensed: rent the hardware, pin the environment, run identical workloads, publish the failures as data, and never quote a number you didn't produce. It's slower than scraping and smaller than aggregating, 51 cards instead of 453, but every one of those numbers is comparable to every other, and I can tell you exactly how each was made. That's the trade this site is built on.