NVIDIA RTX PRO 6000 Blackwell
GPUBlackwell; 96GB VRAM, 1792 GB/s bandwidth, 600W TDP, CUDA cores 24,064. Release date: 2025-03-18. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.
11
Run records
4
Models covered
4
Frameworks covered
L0 Self-reported × 11
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- vLLM · FP16 Decode 26.1 · Prefill 195.8 ·
- vLLM · Q8_0 Decode 52.2 · Prefill 391.5 ·
- vLLM · Q5_K_M Decode 76.2 · Prefill 571.5 ·
- llama.cpp · FP16 Decode 24 · Prefill 180 ·
- llama.cpp · Q8_0 Decode 47.9 · Prefill 359.3 ·
- llama.cpp · Q5_K_M Decode 69.9 · Prefill 524.3 ·
- vLLM · IQ3_S Decode 31.9 · Prefill 239.3 ·
- vLLM · IQ2_XS Decode 38.3 · Prefill 287.3 ·
- vLLM · IQ3_S Decode 5 · Prefill 37.5 ·
- SGLang · NVFP4 Decode 171.1 · Prefill 10877.7 ·
- Diffusers · BF16 Decode — · Prefill — ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- NVIDIA
- Architecture
- Blackwell
- VRAM
- 96 GB
- Memory bandwidth
- 1792 GB/s
- TDP
- 600 W
- Release date
- 2025-03-18
Run records
11 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.8-27B | vLLM | FP16 | 26.1 tok/s | — | 1 |
| Qwen3.8-27B | vLLM | Q8_0 | 52.2 tok/s | — | 1 |
| Qwen3.8-27B | vLLM | Q5_K_M | 76.2 tok/s | — | 1 |
| Qwen3.8-27B | llama.cpp | FP16 | 24 tok/s | — | 1 |
| Qwen3.8-27B | llama.cpp | Q8_0 | 47.9 tok/s | — | 1 |
| Qwen3.8-27B | llama.cpp | Q5_K_M | 69.9 tok/s | — | 1 |
| Laguna-S-2.1 | vLLM | IQ3_S | 31.9 tok/s | — | 1 |
| Laguna-S-2.1 | vLLM | IQ2_XS | 38.3 tok/s | — | 1 |
| GLM-5.3 | vLLM | IQ3_S | 5 tok/s | — | 1 |
| Qwen3.8-27B | SGLang | NVFP4 | 171.1 tok/s | — | 1 |
| MiniMax-H3 | Diffusers | BF16 | — | 317 s | 1 |