2x AMD Radeon RX 7900 XTX
GPU16
Run records
1
Models covered
4
Frameworks covered
L2 Cross-framework verified × 16
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ4_XS Decode 108 · Prefill 939 ·
- llama.cpp · Q4_K_M Decode 98 · Prefill 892 ·
- llama.cpp · Q5_K_M Decode 80 · Prefill 948 ·
- llama.cpp · Q8_0 Decode 66 · Prefill 724 ·
- vLLM · IQ4_XS Decode 82 · Prefill 946 ·
- vLLM · Q4_K_M Decode 73 · Prefill 855 ·
- vLLM · Q5_K_M Decode 75 · Prefill 843 ·
- vLLM · Q8_0 Decode 59 · Prefill 489 ·
- Ollama · IQ4_XS Decode 94 · Prefill 1095 ·
- Ollama · Q4_K_M Decode 87 · Prefill 997 ·
- Ollama · Q5_K_M Decode 89 · Prefill 724 ·
- Ollama · Q8_0 Decode 59 · Prefill 668 ·
- Strata · IQ4_XS Decode 152 · Prefill 1362 ·
- Strata · Q4_K_M Decode 139 · Prefill 1188 ·
- Strata · Q5_K_M Decode 121 · Prefill 1339 ·
- Strata · Q8_0 Decode 87 · Prefill 1027 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- AMD
- VRAM
- 48 GB
- Memory bandwidth
- 1920 GB/s
Run records
16 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-27B | llama.cpp | IQ4_XS | 108 tok/s | — | 1 |
| Qwen3.5-27B | llama.cpp | Q4_K_M | 98 tok/s | — | 1 |
| Qwen3.5-27B | llama.cpp | Q5_K_M | 80 tok/s | — | 1 |
| Qwen3.5-27B | llama.cpp | Q8_0 | 66 tok/s | — | 1 |
| Qwen3.5-27B | vLLM | IQ4_XS | 82 tok/s | — | 1 |
| Qwen3.5-27B | vLLM | Q4_K_M | 73 tok/s | — | 1 |
| Qwen3.5-27B | vLLM | Q5_K_M | 75 tok/s | — | 1 |
| Qwen3.5-27B | vLLM | Q8_0 | 59 tok/s | — | 1 |
| Qwen3.5-27B | Ollama | IQ4_XS | 94 tok/s | — | 1 |
| Qwen3.5-27B | Ollama | Q4_K_M | 87 tok/s | — | 1 |
| Qwen3.5-27B | Ollama | Q5_K_M | 89 tok/s | — | 1 |
| Qwen3.5-27B | Ollama | Q8_0 | 59 tok/s | — | 1 |
| Qwen3.5-27B | Strata | IQ4_XS | 152 tok/s | — | 1 |
| Qwen3.5-27B | Strata | Q4_K_M | 139 tok/s | — | 1 |
| Qwen3.5-27B | Strata | Q5_K_M | 121 tok/s | — | 1 |
| Qwen3.5-27B | Strata | Q8_0 | 87 tok/s | — | 1 |