Training throughput in tokens per second: LoRA on a fixed batch, the number that decides whether a fine-tune is an evening or a week.
Weights: Qwen/Qwen2.5-1.5B
Measured in train tok/s. Longer is faster.
| # | GPU | train tok/s |
|---|---|---|
| 1 | NVIDIA B300 | 22351.1 |
| 2 | NVIDIA H100 80GB HBM3 | 16034.9 |
| 3 | NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 15753.9 |
| 4 | NVIDIA B200 | 15322 |
| 5 | NVIDIA H200 | 14931.7 |
| 6 | NVIDIA L40S | 12120.1 |
| 7 | NVIDIA A100 80GB SXM4 | 8252.8 |
| 8 | NVIDIA A100 40GB SXM4 | 6349.7 |
| 9 | NVIDIA A10G | 4214.4 |
| 10 | NVIDIA L4 | 3720.9 |
| 11 | NVIDIA T4 | 1434.4 |
Weights: HuggingFaceTB/SmolLM2-1.7B
Measured in train tok/s. Longer is faster.
| # | GPU | train tok/s |
|---|---|---|
| 1 | NVIDIA B300 | 26114.3 |
| 2 | NVIDIA H100 80GB HBM3 | 18910.3 |
| 3 | NVIDIA B200 | 18064.7 |
| 4 | NVIDIA H200 | 17650.4 |
| 5 | NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 16255.4 |
| 6 | NVIDIA L40S | 12052.1 |
| 7 | NVIDIA A100 80GB SXM4 | 9750.8 |
| 8 | NVIDIA A100 40GB SXM4 | 6938.8 |
| 9 | NVIDIA A10G | 3668.3 |
| 10 | NVIDIA L4 | 3240.2 |
| 11 | NVIDIA T4 | 1561.9 |
Weights: TinyLlama/TinyLlama-1.1B-Chat-v1.0
Measured in train tok/s. Longer is faster.
| # | GPU | train tok/s |
|---|---|---|
| 1 | NVIDIA B300 | 26938.2 |
| 2 | NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 18556.4 |
| 3 | NVIDIA B200 | 17993.5 |
| 4 | NVIDIA H100 80GB HBM3 | 17522 |
| 5 | NVIDIA L40S | 16408.1 |
| 6 | NVIDIA H200 | 16222.4 |
| 7 | NVIDIA A100 80GB SXM4 | 9553.3 |
| 8 | NVIDIA A100 40GB SXM4 | 7471 |
| 9 | NVIDIA A10G | 5499.6 |
| 10 | NVIDIA L4 | 4756 |
| 11 | NVIDIA T4 | 1520.1 |
Weights: Qwen/Qwen2.5-7B
Measured in train tok/s. Longer is faster.
| # | GPU | train tok/s |
|---|---|---|
| 1 | NVIDIA B300 | 14323.1 |
| 2 | NVIDIA B200 | 13722 |
| 3 | NVIDIA H200 | 8855.4 |
| 4 | NVIDIA H100 80GB HBM3 | 8514.3 |
| 5 | NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 6041.8 |
| 6 | NVIDIA A100 80GB SXM4 | 3900.6 |
| 7 | NVIDIA L40S | 3736 |
| 8 | NVIDIA A100 40GB SXM4 | 3670.8 |
| 9 | NVIDIA A10G | 1239 |
| 10 | NVIDIA L4 | 1063.9 |
Every number is a first-party run on our own bench. Cards missing from a board have not been measured on that model yet, or cannot fit it.