NVIDIA RTX 5080
GPUNVIDIA Blackwell (GB203): 16 GB GDDR7 on a 256-bit bus, 960 GB/s of memory bandwidth, 360 W board power.
16
Run records
6
Models covered
4
Frameworks covered
L2 Cross-framework verified × 16
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ4_XS Decode 133.14 · Prefill 731.5 ·
- Strata · Q3_K_M Decode 231.73 · Prefill 785.76 ·
- llama.cpp · IQ2_M Decode 338.41 · Prefill 1613.66 ·
- Strata · FP16 Decode 178.36 · Prefill 737.11 ·
- Strata · Q4_K_S Decode 363.26 · Prefill 1495.67 ·
- llama.cpp · IQ4_XS Decode 97.14 · Prefill 496.38 ·
- llama.cpp · Q3_K_M Decode 108.92 · Prefill 455.22 ·
- vLLM · IQ2_M Decode 131.31 · Prefill 559.09 ·
- llama.cpp · Q5_K_S Decode 91.75 · Prefill 454.66 ·
- Ollama · Q3_K_L Decode 101.88 · Prefill 405.42 ·
- Strata · IQ4_XS Decode 144.31 · Prefill 545.08 ·
- llama.cpp · Q4_K_S Decode 194.19 · Prefill 976.97 ·
- llama.cpp · Q8_0 Decode 121.42 · Prefill 409.38 ·
- Ollama · Q3_K_M Decode 198.41 · Prefill 879.72 ·
- Ollama · Q8_0 Decode 116.38 · Prefill 555.86 ·
- Ollama · IQ2_XS Decode 103.79 · Prefill 371.24 ·
Key specs
- Vendor
- NVIDIA
- Architecture
- Blackwell
- VRAM
- 16 GB
- Memory bandwidth
- 960 GB/s
- TDP
- 360 W
- Release date
- 2025-01-30
Run records
16 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-4B | llama.cpp | IQ4_XS | 133.1 tok/s | — | 1 |
| Qwen3.5-4B | Strata | Q3_K_M | 231.7 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | IQ2_M | 338.4 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | FP16 | 178.4 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | Q4_K_S | 363.3 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | IQ4_XS | 97.1 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q3_K_M | 108.9 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | IQ2_M | 131.3 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q5_K_S | 91.8 tok/s | — | 1 |
| Qwen3-8B | Ollama | Q3_K_L | 101.9 tok/s | — | 1 |
| Qwen3-8B | Strata | IQ4_XS | 144.3 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | Q4_K_S | 194.2 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | Q8_0 | 121.4 tok/s | — | 1 |
| Qwen3-1.7B | Ollama | Q3_K_M | 198.4 tok/s | — | 1 |
| Qwen3-1.7B | Ollama | Q8_0 | 116.4 tok/s | — | 1 |
| gemma-4-12B-it | Ollama | IQ2_XS | 103.8 tok/s | — | 1 |