NVIDIA RTX PRO 5000 Blackwell
GPUBlackwell; 48GB VRAM, 1344 GB/s bandwidth, 300W TDP, CUDA cores 14,080. Release date: 2025-03-18. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.
9
Run records
3
Models covered
2
Frameworks covered
L0 Self-reported × 9
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- SGLang · IQ2_XS Decode 4.3 · Prefill 32.3 ·
- SGLang · IQ3_S Decode 3.6 · Prefill 27 ·
- vLLM · IQ2_XS Decode 4.5 · Prefill 33.8 ·
- vLLM · IQ3_S Decode 3.7 · Prefill 27.8 ·
- SGLang · IQ2_XS Decode 4.3 · Prefill 32.3 ·
- SGLang · IQ3_S Decode 3.6 · Prefill 27 ·
- vLLM · IQ2_XS Decode 4.4 · Prefill 33 ·
- vLLM · IQ3_S Decode 3.7 · Prefill 27.8 ·
- SGLang · FP8 Decode 31.5 · Prefill 5089 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- NVIDIA
- Architecture
- Blackwell
- VRAM
- 48 GB
- Memory bandwidth
- 1344 GB/s
- TDP
- 300 W
- Release date
- 2025-03-18
Run records
9 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| GLM-5.3 | SGLang | IQ2_XS | 4.3 tok/s | — | 1 |
| GLM-5.3 | SGLang | IQ3_S | 3.6 tok/s | — | 1 |
| GLM-5.3 | vLLM | IQ2_XS | 4.5 tok/s | — | 1 |
| GLM-5.3 | vLLM | IQ3_S | 3.7 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | SGLang | IQ2_XS | 4.3 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | SGLang | IQ3_S | 3.6 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | vLLM | IQ2_XS | 4.4 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | vLLM | IQ3_S | 3.7 tok/s | — | 1 |
| Qwen3.8-27B | SGLang | FP8 | 31.5 tok/s | — | 1 |