AMD Radeon RX 9070 XT
GPUAMD RDNA 4 (Navi 48): 16 GB GDDR6 on a 256-bit bus, ~644.6 GB/s of memory bandwidth, 304 W board power.
56
Run records
17
Models covered
4
Frameworks covered
L2 Cross-framework verified × 56
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ2_M Decode 196.42 · Prefill 767.31 ·
- Ollama · FP16 Decode 69.03 · Prefill 237.27 ·
- Ollama · Q6_K Decode 103.08 · Prefill 389.31 ·
- vLLM · Q5_K_S Decode 285.41 · Prefill 1428.43 ·
- Ollama · FP16 Decode 127.98 · Prefill 427.95 ·
- Ollama · Q5_K_S Decode 242.84 · Prefill 980.43 ·
- llama.cpp · Q3_K_L Decode 114.6 · Prefill 612.55 ·
- Ollama · Q4_K_M Decode 103.73 · Prefill 474.73 ·
- llama.cpp · Q4_K_S Decode 123.04 · Prefill 487.34 ·
- vLLM · Q2_K Decode 168.21 · Prefill 814.55 ·
- llama.cpp · Q2_K Decode 126.21 · Prefill 558.74 ·
- Strata · IQ3_XS Decode 157.74 · Prefill 536.72 ·
- llama.cpp · IQ4_XS Decode 135.74 · Prefill 591.43 ·
- vLLM · Q3_K_S Decode 162.87 · Prefill 574.19 ·
- vLLM · Q4_K_M Decode 144.21 · Prefill 558.74 ·
- llama.cpp · Q5_K_S Decode 108.87 · Prefill 379.25 ·
- Ollama · Q2_K Decode 175.64 · Prefill 836.53 ·
- llama.cpp · IQ3_XS Decode 319.75 · Prefill 1663.47 ·
- Strata · IQ3_S Decode 308.2 · Prefill 1415.95 ·
- Strata · IQ4_XS Decode 311.94 · Prefill 1321.43 ·
- llama.cpp · IQ2_XS Decode 150.94 · Prefill 517.81 ·
- llama.cpp · IQ3_XS Decode 118.99 · Prefill 614.09 ·
- llama.cpp · Q5_K_M Decode 84.68 · Prefill 344.07 ·
- vLLM · IQ4_XS Decode 107.13 · Prefill 359.17 ·
- Ollama · Q3_K_L Decode 133.65 · Prefill 600.15 ·
- vLLM · IQ1_S Decode 133.24 · Prefill 623.53 ·
- vLLM · Q3_K_S Decode 113.68 · Prefill 457.77 ·
- Ollama · IQ2_XS Decode 124.46 · Prefill 417.76 ·
- llama.cpp · Q5_K_M Decode 182.06 · Prefill 1000.27 ·
- vLLM · IQ3_S Decode 301.46 · Prefill 1013.4 ·
- llama.cpp · IQ3_S Decode 102.09 · Prefill 492.72 ·
- Ollama · IQ3_XS Decode 105.64 · Prefill 538.49 ·
- Ollama · Q4_K_S Decode 80.27 · Prefill 324.54 ·
- vLLM · IQ2_XS Decode 142.91 · Prefill 782.4 ·
- Strata · IQ2_XS Decode 192.63 · Prefill 774.46 ·
- llama.cpp · Q2_K Decode 162 · Prefill 560.55 ·
- llama.cpp · Q6_K Decode 94.45 · Prefill 349.72 ·
- Ollama · IQ3_S Decode 124.75 · Prefill 481.56 ·
- Ollama · IQ2_M Decode 219.34 · Prefill 1047.84 ·
- Ollama · IQ2_XS Decode 223.89 · Prefill 1223.25 ·
- vLLM · IQ2_XS Decode 186.6 · Prefill 815.85 ·
- vLLM · Q5_K_S Decode 103.28 · Prefill 532.39 ·
- Ollama · Q3_K_M Decode 127.72 · Prefill 433.24 ·
- Ollama · Q3_K_S Decode 126.74 · Prefill 598.67 ·
- llama.cpp · Q5_K_S Decode 267.22 · Prefill 915.84 ·
- Strata · IQ2_M Decode 311.28 · Prefill 1001.12 ·
- Strata · IQ3_S Decode 358.46 · Prefill 1486.41 ·
- llama.cpp · IQ3_S Decode 181.8 · Prefill 969.43 ·
- llama.cpp · Q3_K_M Decode 169.43 · Prefill 689.96 ·
- Strata · Q2_K Decode 287.44 · Prefill 1250.58 ·
Key specs
- Vendor
- AMD
- Architecture
- RDNA 4
- VRAM
- 16 GB
- Memory bandwidth
- 645 GB/s
- TDP
- 304 W
- Release date
- 2025-03-06
Run records
50 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-4B | llama.cpp | IQ2_M | 196.4 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | FP16 | 69 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q6_K | 103.1 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | Q5_K_S | 285.4 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | FP16 | 128 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | Q5_K_S | 242.8 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q3_K_L | 114.6 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q4_K_M | 103.7 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q4_K_S | 123 tok/s | — | 1 |
| Qwen3-8B | vLLM | Q2_K | 168.2 tok/s | — | 1 |
| gemma-4-12B-it | llama.cpp | Q2_K | 126.2 tok/s | — | 1 |
| gemma-4-12B-it | Strata | IQ3_XS | 157.7 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | IQ4_XS | 135.7 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q3_K_S | 162.9 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q4_K_M | 144.2 tok/s | — | 1 |
| Llama-3-8B-Instruct | llama.cpp | Q5_K_S | 108.9 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | Q2_K | 175.6 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | llama.cpp | IQ3_XS | 319.8 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Strata | IQ3_S | 308.2 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Strata | IQ4_XS | 311.9 tok/s | — | 1 |
| DeepSeek-R1-Distill-Qwen-14B | llama.cpp | IQ2_XS | 150.9 tok/s | — | 1 |
| Kimi-K3 | llama.cpp | IQ3_XS | 119 tok/s | — | 1 |
| Kimi-K3 | llama.cpp | Q5_K_M | 84.7 tok/s | — | 1 |
| Kimi-K3 | vLLM | IQ4_XS | 107.1 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q3_K_L | 133.7 tok/s | — | 1 |
| Qwen3.5-9B | vLLM | IQ1_S | 133.2 tok/s | — | 1 |
| Qwen3.5-9B | vLLM | Q3_K_S | 113.7 tok/s | — | 1 |
| Qwen3.5-9B | Ollama | IQ2_XS | 124.5 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | Q5_K_M | 182.1 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | IQ3_S | 301.5 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | IQ3_S | 102.1 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | IQ3_XS | 105.6 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q4_K_S | 80.3 tok/s | — | 1 |
| Qwen3-8B | vLLM | IQ2_XS | 142.9 tok/s | — | 1 |
| Qwen3-8B | Strata | IQ2_XS | 192.6 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q2_K | 162 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q6_K | 94.5 tok/s | — | 1 |
| Qwen3-4B | Ollama | IQ3_S | 124.8 tok/s | — | 1 |
| Qwen3-1.7B | Ollama | IQ2_M | 219.3 tok/s | — | 1 |
| Qwen3-1.7B | Ollama | IQ2_XS | 223.9 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | IQ2_XS | 186.6 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | Q5_K_S | 103.3 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | Ollama | Q3_K_M | 127.7 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | Ollama | Q3_K_S | 126.7 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | llama.cpp | Q5_K_S | 267.2 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Strata | IQ2_M | 311.3 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Strata | IQ3_S | 358.5 tok/s | — | 1 |
| Qwen2.5-Omni-3B | llama.cpp | IQ3_S | 181.8 tok/s | — | 1 |
| Qwen2.5-Omni-3B | llama.cpp | Q3_K_M | 169.4 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Strata | Q2_K | 287.4 tok/s | — | 1 |