NVIDIA RTX PRO 6000 Blackwell

GPU

Blackwell; 96GB VRAM, 1792 GB/s bandwidth, 600W TDP, CUDA cores 24,064. Release date: 2025-03-18. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.

11

Run records

4

Models covered

4

Frameworks covered

L0 Self-reported × 11

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. vLLM · FP16 Decode 26.1 · Prefill 195.8 ·
  2. vLLM · Q8_0 Decode 52.2 · Prefill 391.5 ·
  3. vLLM · Q5_K_M Decode 76.2 · Prefill 571.5 ·
  4. llama.cpp · FP16 Decode 24 · Prefill 180 ·
  5. llama.cpp · Q8_0 Decode 47.9 · Prefill 359.3 ·
  6. llama.cpp · Q5_K_M Decode 69.9 · Prefill 524.3 ·
  7. vLLM · IQ3_S Decode 31.9 · Prefill 239.3 ·
  8. vLLM · IQ2_XS Decode 38.3 · Prefill 287.3 ·
  9. vLLM · IQ3_S Decode 5 · Prefill 37.5 ·
  10. SGLang · NVFP4 Decode 171.1 · Prefill 10877.7 ·
  11. Diffusers · BF16 Decode — · Prefill — ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Blackwell
VRAM
96 GB
Memory bandwidth
1792 GB/s
TDP
600 W
Release date
2025-03-18

Run records

11 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.8-27BvLLMFP1626.1 tok/s—1
Qwen3.8-27BvLLMQ8_052.2 tok/s—1
Qwen3.8-27BvLLMQ5_K_M76.2 tok/s—1
Qwen3.8-27Bllama.cppFP1624 tok/s—1
Qwen3.8-27Bllama.cppQ8_047.9 tok/s—1
Qwen3.8-27Bllama.cppQ5_K_M69.9 tok/s—1
Laguna-S-2.1vLLMIQ3_S31.9 tok/s—1
Laguna-S-2.1vLLMIQ2_XS38.3 tok/s—1
GLM-5.3vLLMIQ3_S5 tok/s—1
Qwen3.8-27BSGLangNVFP4171.1 tok/s—1
MiniMax-H3DiffusersBF16—317 s1