AMD Radeon RX 9070
GPUAMD RDNA 4 (Navi 48): 16 GB GDDR6 on a 256-bit bus, ~644.6 GB/s of memory bandwidth, 220 W board power. Same die as the RX 9070 XT at lower power.
57
Run records
17
Models covered
4
Frameworks covered
L2 Cross-framework verified × 57
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ2_XS Decode 204.13 · Prefill 1113.36 ·
- llama.cpp · Q3_K_L Decode 162.78 · Prefill 836.11 ·
- vLLM · Q3_K_S Decode 194.22 · Prefill 736.68 ·
- vLLM · Q4_K_M Decode 137.14 · Prefill 684.52 ·
- llama.cpp · Q3_K_S Decode 282.09 · Prefill 1248.73 ·
- llama.cpp · Q4_K_M Decode 250.59 · Prefill 955.15 ·
- Ollama · IQ1_S Decode 353.51 · Prefill 1875.31 ·
- llama.cpp · Q3_K_L Decode 112.13 · Prefill 574.57 ·
- llama.cpp · Q5_K_M Decode 84.7 · Prefill 430.21 ·
- Strata · Q4_K_S Decode 150.11 · Prefill 526.56 ·
- vLLM · IQ3_XS Decode 114.22 · Prefill 439.54 ·
- vLLM · IQ1_S Decode 209.03 · Prefill 1072.51 ·
- vLLM · Q3_K_M Decode 141.43 · Prefill 703.16 ·
- Strata · Q3_K_L Decode 194.02 · Prefill 811.8 ·
- llama.cpp · Q5_K_S Decode 113.29 · Prefill 563 ·
- llama.cpp · Q4_K_M Decode 313.76 · Prefill 1093.15 ·
- llama.cpp · Q4_K_S Decode 290.99 · Prefill 1343.85 ·
- vLLM · IQ4_XS Decode 366.03 · Prefill 1680.79 ·
- vLLM · Q3_K_S Decode 348.42 · Prefill 1407.74 ·
- vLLM · IQ4_XS Decode 102.59 · Prefill 523.94 ·
- vLLM · Q3_K_M Decode 122.09 · Prefill 529.79 ·
- Ollama · IQ3_XS Decode 117.33 · Prefill 604.75 ·
- Ollama · Q2_K Decode 125.3 · Prefill 531.07 ·
- llama.cpp · IQ4_XS Decode 127.63 · Prefill 638.8 ·
- vLLM · IQ1_S Decode 172.53 · Prefill 556.88 ·
- vLLM · IQ3_S Decode 146.59 · Prefill 752.13 ·
- Ollama · Q2_K Decode 96.63 · Prefill 444.09 ·
- Ollama · Q4_K_S Decode 80.57 · Prefill 434.52 ·
- Strata · Q4_K_S Decode 115.51 · Prefill 577.32 ·
- llama.cpp · Q3_K_L Decode 236.42 · Prefill 1273.33 ·
- vLLM · IQ1_S Decode 361.98 · Prefill 1724.43 ·
- vLLM · IQ2_M Decode 325.3 · Prefill 1252.35 ·
- llama.cpp · Q4_K_M Decode 78.36 · Prefill 376.38 ·
- llama.cpp · Q4_K_S Decode 79.97 · Prefill 402.87 ·
- Ollama · Q4_K_S Decode 82.2 · Prefill 391.04 ·
- Strata · Q3_K_L Decode 126.35 · Prefill 621.6 ·
- Strata · Q3_K_S Decode 136.8 · Prefill 667.03 ·
- llama.cpp · Q2_K Decode 140.23 · Prefill 653.54 ·
- llama.cpp · Q8_0 Decode 73.3 · Prefill 246.97 ·
- Strata · IQ4_XS Decode 168.75 · Prefill 844.33 ·
- llama.cpp · IQ1_S Decode 251.73 · Prefill 1075.85 ·
- llama.cpp · Q3_K_L Decode 168.28 · Prefill 901.02 ·
- Strata · IQ3_XS Decode 263.09 · Prefill 972.27 ·
- Strata · Q5_K_M Decode 186.72 · Prefill 727.73 ·
- vLLM · IQ3_S Decode 127.52 · Prefill 534.05 ·
- vLLM · Q3_K_S Decode 140.19 · Prefill 769.84 ·
- Ollama · IQ1_S Decode 375.3 · Prefill 1369.71 ·
- Ollama · Q6_K Decode 169 · Prefill 555.64 ·
- vLLM · Q4_K_S Decode 147.81 · Prefill 510.18 ·
- vLLM · Q8_0 Decode 91.32 · Prefill 374.63 ·
Key specs
- Vendor
- AMD
- Architecture
- RDNA 4
- VRAM
- 16 GB
- Memory bandwidth
- 645 GB/s
- TDP
- 220 W
- Release date
- 2025-03-06
Run records
50 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-4B | llama.cpp | IQ2_XS | 204.1 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | Q3_K_L | 162.8 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q3_K_S | 194.2 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q4_K_M | 137.1 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | Q3_K_S | 282.1 tok/s | — | 2 |
| Qwen3.5-0.8B | llama.cpp | Q4_K_M | 250.6 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | IQ1_S | 353.5 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q3_K_L | 112.1 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q5_K_M | 84.7 tok/s | — | 1 |
| Qwen3-8B | Strata | Q4_K_S | 150.1 tok/s | — | 1 |
| gemma-4-12B-it | vLLM | IQ3_XS | 114.2 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | IQ1_S | 209 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q3_K_M | 141.4 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Strata | Q3_K_L | 194 tok/s | — | 1 |
| Llama-3-8B-Instruct | llama.cpp | Q5_K_S | 113.3 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | llama.cpp | Q4_K_M | 313.8 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | llama.cpp | Q4_K_S | 291 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | vLLM | IQ4_XS | 366 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | vLLM | Q3_K_S | 348.4 tok/s | — | 1 |
| Kimi-K3 | vLLM | IQ4_XS | 102.6 tok/s | — | 1 |
| Kimi-K3 | vLLM | Q3_K_M | 122.1 tok/s | — | 1 |
| Kimi-K3 | Ollama | IQ3_XS | 117.3 tok/s | — | 1 |
| Kimi-K3 | Ollama | Q2_K | 125.3 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | IQ4_XS | 127.6 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | IQ1_S | 172.5 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | IQ3_S | 146.6 tok/s | — | 1 |
| Qwen3.5-9B | Ollama | Q2_K | 96.6 tok/s | — | 1 |
| Qwen3.5-9B | Ollama | Q4_K_S | 80.6 tok/s | — | 1 |
| Qwen3.5-9B | Strata | Q4_K_S | 115.5 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | Q3_K_L | 236.4 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | IQ1_S | 362 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | IQ2_M | 325.3 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q4_K_M | 78.4 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q4_K_S | 80 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q4_K_S | 82.2 tok/s | — | 1 |
| Qwen3-8B | Strata | Q3_K_L | 126.3 tok/s | — | 1 |
| Qwen3-8B | Strata | Q3_K_S | 136.8 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q2_K | 140.2 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q8_0 | 73.3 tok/s | — | 1 |
| Qwen3-4B | Strata | IQ4_XS | 168.8 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | IQ1_S | 251.7 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | Q3_K_L | 168.3 tok/s | — | 1 |
| Qwen3-1.7B | Strata | IQ3_XS | 263.1 tok/s | — | 1 |
| Qwen3-1.7B | Strata | Q5_K_M | 186.7 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | IQ3_S | 127.5 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | Q3_K_S | 140.2 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Ollama | IQ1_S | 375.3 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Ollama | Q6_K | 169 tok/s | — | 1 |
| Qwen2.5-Omni-3B | vLLM | Q4_K_S | 147.8 tok/s | — | 1 |
| Qwen2.5-Omni-3B | vLLM | Q8_0 | 91.3 tok/s | — | 1 |