GPU Benchmarks
Compare performance across our GPU lineup to find the perfect balance of power and cost for your workloads.
Deploy a GPU NowHow we benchmark our GPUs
Our benchmarks are designed to give you a real-world understanding of how each GPU performs across common AI workloads. All values are normalized to the H100 SXM5 performance (100%), so you can easily compare relative performance.
Inference Benchmarks
- Llama2 70B: Tokens per second for Llama2 70B model inference
- Mistral 7B: Tokens per second for Mistral 7B model inference
- Stable Diffusion: Images per minute for 512x512 resolution with 50 steps
Training Benchmarks
- Training Performance: Composite score based on training throughput across multiple model architectures
- All benchmarks are run on identical system configurations with the same software stack
- Price/Performance: Hourly price divided by performance score
For custom benchmarks or detailed performance data for your specific workload, please contact our team.
Performance Comparison
| GPU Model | Price/hour | Llama2 70B (tokens/sec) | Mistral 7B (tokens/sec) | Stable Diffusion (imgs/min) | Training (composite) | Action |
|---|---|---|---|---|---|---|
| H100 SXM5 80GB | $2.25 | 100.0 | 100.0 | 100.0 | 100.0 | Deploy |
| A100 SXM4 80GB | $1.80 | 72.3 | 86.5 | 78.2 | 80.5 | Deploy |
| A100 PCIe 80GB | $1.50 | 64.8 | 75.3 | 71.9 | 72.1 | Deploy |
| L40 48GB | $0.95 | 42.7 | 58.9 | 60.5 | 55.2 | Deploy |
| RTX 4090 24GB | $0.35 | 39.8 | 57.2 | 68.3 | 51.6 | Deploy |
| RTX 3090 24GB | $0.20 | 28.5 | 44.1 | 52.7 | 39.3 | Deploy |
Last Updated: July 24, 2024. All performance values normalized to H100 SXM5 (100%).
Price-to-Performance Ratio
Lower is better - Price per hour divided by performance score
Value Leaders by Workload
For Large Model Inference (Llama2 70B)
- 1. RTX 3090: Best Value$0.007/point
- 2. RTX 4090: Great Value$0.009/point
- 3. L40: Good Value$0.022/point
For Image Generation (Stable Diffusion)
- 1. RTX 3090: Best Value$0.004/point
- 2. RTX 4090: Great Value$0.005/point
- 3. L40: Good Value$0.016/point