Pairing a small draft model with a large one is sold as a free speedup. Measured per card, it is often a slowdown. Above 1.0x it helped, below 1.0x it cost you.
Measured in x vs solo. Longer is faster.
| # | GPU | x vs solo |
|---|---|---|
| 1 | NVIDIA L4 | 0.97 |
| 2 | NVIDIA L40S | 0.95 |
| 3 | NVIDIA A10G | 0.92 |
| 4 | NVIDIA T4 | 0.9 |
| 5 | NVIDIA H100 80GB HBM3 | 0.73 |
| 6 | NVIDIA A100 40GB SXM4 | 0.69 |
Every number is a first-party run on our own bench. Cards missing from a board have not been measured on that model yet, or cannot fit it.