AMD Radeon RX 7900 XTX
GPUAMD RDNA 3 flagship (Navi 31, chiplet design): 24 GB GDDR6 on a 384-bit bus, 960 GB/s of memory bandwidth, 355 W board power.
89
Run records
18
Models covered
4
Frameworks covered
L0 Self-reported × 7 · L2 Cross-framework verified × 82
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- vLLM · IQ3_XS Decode 133.18 · Prefill 447.04 ·
- Ollama · Q5_K_M Decode 71.28 · Prefill 362.47 ·
- vLLM · Q5_K_M Decode 70.36 · Prefill 357.32 ·
- Ollama · IQ3_S Decode 86.72 · Prefill 439.93 ·
- vLLM · Q5_K_S Decode 190.47 · Prefill 971.18 ·
- Ollama · Q3_K_L Decode 216.92 · Prefill 1042.56 ·
- llama.cpp · Q4_K_M Decode 102.56 · Prefill 353.32 ·
- Ollama · Q4_K_M Decode 86.87 · Prefill 421.48 ·
- vLLM · Q3_K_M Decode 94.98 · Prefill 305.39 ·
- vLLM · IQ1_S Decode 68.56 · Prefill 341.84 ·
- vLLM · Q4_K_M Decode 100.76 · Prefill 500.94 ·
- vLLM · Q4_K_S Decode 113.75 · Prefill 494.84 ·
- Ollama · Q3_K_L Decode 108.14 · Prefill 521.92 ·
- Ollama · Q3_K_M Decode 103.32 · Prefill 530.53 ·
- llama.cpp · IQ2_XS Decode 280.58 · Prefill 1056.43 ·
- Ollama · Q4_K_M Decode 171.2 · Prefill 687.99 ·
- llama.cpp · Q3_K_L Decode 79 · Prefill 351.53 ·
- Ollama · Q3_K_S Decode 81.43 · Prefill 433.18 ·
- Ollama · Q8_0 Decode 45.02 · Prefill 207.48 ·
- llama.cpp · Q5_K_S Decode 73.41 · Prefill 264.95 ·
- vLLM · IQ2_M Decode 127.88 · Prefill 585.23 ·
- vLLM · Q2_K Decode 108.21 · Prefill 417.97 ·
- vLLM · Q3_K_M Decode 77.87 · Prefill 405.9 ·
- Ollama · IQ1_S Decode 92.04 · Prefill 422.92 ·
- llama.cpp · Q3_K_S Decode 102.17 · Prefill 424.1 ·
- llama.cpp · Q4_K_M Decode 83.47 · Prefill 411.3 ·
- vLLM · Q3_K_M Decode 114.54 · Prefill 374.54 ·
- vLLM · Q4_K_M Decode 87.13 · Prefill 426.78 ·
- vLLM · Q4_K_M Decode 95.65 · Prefill 325.43 ·
- Ollama · Q5_K_S Decode 74.72 · Prefill 318.2 ·
- vLLM · Q6_K Decode 175.87 · Prefill 672.51 ·
- Ollama · Q3_K_L Decode 237.89 · Prefill 1168.16 ·
- llama.cpp · Q4_K_M Decode 65.83 · Prefill 346.39 ·
- Ollama · IQ2_XS Decode 90.76 · Prefill 372.69 ·
- llama.cpp · Q3_K_S Decode 122.17 · Prefill 596.11 ·
- Strata · IQ3_S Decode 166.71 · Prefill 815.55 ·
- Strata · IQ1_S Decode 136.67 · Prefill 575.25 ·
- Ollama · Q5_K_S Decode 148.35 · Prefill 573.18 ·
- Strata · Q4_K_S Decode 289.51 · Prefill 1525.63 ·
- vLLM · Q3_K_L Decode 79.57 · Prefill 292.41 ·
- vLLM · Q3_K_M Decode 85.52 · Prefill 281.83 ·
- Ollama · Q4_K_M Decode 61.26 · Prefill 253.43 ·
- Ollama · Q5_K_M Decode 56.51 · Prefill 294.11 ·
- Ollama · IQ3_XS Decode 90.61 · Prefill 409.37 ·
- Strata · IQ2_M Decode 148.41 · Prefill 586.66 ·
- llama.cpp · IQ2_XS Decode 97.1 · Prefill 483.73 ·
- llama.cpp · Q5_K_M Decode 54.12 · Prefill 201.79 ·
- Strata · IQ2_M Decode 125.31 · Prefill 624.12 ·
- vLLM · IQ4_XS Decode 91.85 · Prefill 357.41 ·
- Strata · IQ4_XS Decode 138.01 · Prefill 527.22 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- AMD
- Architecture
- RDNA 3
- VRAM
- 24 GB
- Memory bandwidth
- 960 GB/s
- TDP
- 355 W
- Release date
- 2022-12-13
Run records
50 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| gemma-4-E4B-it | vLLM | IQ3_XS | 133.2 tok/s | — | 1 |
| gemma-4-E4B-it | Ollama | Q5_K_M | 71.3 tok/s | — | 1 |
| Ministral-3-8B-Instruct-2512 | vLLM | Q5_K_M | 70.4 tok/s | — | 1 |
| Ministral-3-8B-Instruct-2512 | Ollama | IQ3_S | 86.7 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | vLLM | Q5_K_S | 190.5 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | Ollama | Q3_K_L | 216.9 tok/s | — | 1 |
| NVIDIA-Nemotron-3.5-Bolt-E4B-4GB | llama.cpp | Q4_K_M | 102.6 tok/s | — | 1 |
| NVIDIA-Nemotron-3.5-Bolt-E4B-4GB | Ollama | Q4_K_M | 86.9 tok/s | — | 1 |
| elephant-1 | vLLM | Q3_K_M | 95 tok/s | — | 1 |
| Qwen3.8-27B | vLLM | IQ1_S | 68.6 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q4_K_M | 100.8 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q4_K_S | 113.8 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q3_K_L | 108.1 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q3_K_M | 103.3 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | IQ2_XS | 280.6 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | Q4_K_M | 171.2 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q3_K_L | 79 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q3_K_S | 81.4 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q8_0 | 45 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q5_K_S | 73.4 tok/s | — | 1 |
| Qwen3-8B | vLLM | IQ2_M | 127.9 tok/s | — | 2 |
| Qwen3-8B | vLLM | Q2_K | 108.2 tok/s | — | 1 |
| gemma-4-12B-it | vLLM | Q3_K_M | 77.9 tok/s | — | 1 |
| gemma-4-12B-it | Ollama | IQ1_S | 92 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | Q3_K_S | 102.2 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | Q4_K_M | 83.5 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q3_K_M | 114.5 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q4_K_M | 87.1 tok/s | — | 1 |
| Llama-3-8B-Instruct | vLLM | Q4_K_M | 95.7 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | Q5_K_S | 74.7 tok/s | — | 2 |
| Llama-3.2-3B-Instruct | vLLM | Q6_K | 175.9 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Ollama | Q3_K_L | 237.9 tok/s | — | 1 |
| DeepSeek-R1-Distill-Qwen-14B | llama.cpp | Q4_K_M | 65.8 tok/s | — | 1 |
| DeepSeek-R1-Distill-Qwen-14B | Ollama | IQ2_XS | 90.8 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | Q3_K_S | 122.2 tok/s | — | 1 |
| Qwen3.5-4B | Strata | IQ3_S | 166.7 tok/s | — | 1 |
| Qwen3.5-27B | Strata | IQ1_S | 136.7 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | Q5_K_S | 148.3 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | Q4_K_S | 289.5 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | Q3_K_L | 79.6 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | Q3_K_M | 85.5 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q4_K_M | 61.3 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q5_K_M | 56.5 tok/s | — | 1 |
| Qwen3-8B | Ollama | IQ3_XS | 90.6 tok/s | — | 1 |
| Qwen3-8B | Strata | IQ2_M | 148.4 tok/s | — | 1 |
| gemma-4-12B-it | llama.cpp | IQ2_XS | 97.1 tok/s | — | 1 |
| gemma-4-12B-it | llama.cpp | Q5_K_M | 54.1 tok/s | — | 1 |
| gemma-4-12B-it | Strata | IQ2_M | 125.3 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | IQ4_XS | 91.8 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Strata | IQ4_XS | 138 tok/s | — | 1 |