0
Quantized builds
56
Benchmark records
1
Discussions
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
50 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| NVIDIA RTX 4090 | llama.cpp | IQ4_XS | 123.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4090 | Ollama | Q3_K_L | 113.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4090 | Ollama | Q8_0 | 70.6 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 3090 | llama.cpp | Q4_K_M | 61.2 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 3090 | Ollama | Q5_K_M | 54 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4060 Ti 16GB | vLLM | Q5_K_S | 82.8 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4060 Ti 16GB | Ollama | IQ3_XS | 90.7 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4060 Ti 16GB | Ollama | IQ4_XS | 88 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 7900 XTX | llama.cpp | IQ1_S | 109.8 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 7900 XTX | llama.cpp | Q3_K_L | 75 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 7900 XTX | Ollama | Q5_K_S | 56.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5090 | vLLM | Q3_K_L | 253.5 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5090 | vLLM | Q4_K_S | 223.8 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5090 | Strata | IQ3_S | 330.4 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5080 | Ollama | Q5_K_S | 124.2 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5080 | Ollama | Q8_0 | 85.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5080 | Strata | Q8_0 | 128.7 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 3060 12GB | llama.cpp | Q3_K_L | 55.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 3060 12GB | llama.cpp | Q3_K_M | 58.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 3060 12GB | vLLM | IQ4_XS | 53.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 XT | llama.cpp | IQ3_XS | 119 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 XT | llama.cpp | Q5_K_M | 84.7 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 XT | vLLM | IQ4_XS | 107.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 Ti | vLLM | IQ1_S | 131.3 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 Ti | vLLM | IQ3_S | 95.4 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 Ti | Strata | IQ3_S | 132.6 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 Ti | Strata | Q3_K_S | 129.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 | vLLM | Q3_K_L | 76.5 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 | Strata | Q5_K_M | 76.5 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 Super | llama.cpp | IQ2_XS | 184.9 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 Super | Strata | IQ1_S | 255.3 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 | vLLM | IQ4_XS | 102.6 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 | vLLM | Q3_K_M | 122.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 | Ollama | IQ3_XS | 117.3 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| AMD Radeon RX 9070 | Ollama | Q2_K | 125.3 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 | Ollama | IQ3_XS | 139.1 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 | Ollama | Q5_K_M | 91.5 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 | Strata | FP16 | 79.7 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 | Strata | Q4_K_M | 165.8 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4070 Ti | llama.cpp | Q4_K_M | 95.6 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4070 Ti | Ollama | Q3_K_S | 101.2 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| Intel Arc B580 | llama.cpp | IQ3_S | 47 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| Intel Arc B580 | llama.cpp | Q3_K_L | 43.5 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| Intel Arc B580 | Strata | IQ2_XS | 79 tok/s | 1.05 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5070 | SGLang | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | SGLang | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | SGLang | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | SGLang | Q4_K_M | — | — | 1 | L0 Self-reported |
56 of these records are not linked to a specific quantized build; the full set is on the search page.