0
Quantized builds
20
Benchmark records
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
20 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| NVIDIA RTX 5070 SUPER | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 SUPER | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 SUPER | llama.cpp | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 SUPER | llama.cpp | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 3090 | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 3090 | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 3090 | llama.cpp | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 3090 | llama.cpp | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | llama.cpp | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | llama.cpp | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | vLLM | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | vLLM | Q4_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q5_K_M | — | — | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q4_K_M | — | — | 1 | L0 Self-reported |
20 of these records are not linked to a specific quantized build; the full set is on the search page.
Community discussion
No discussions yet — start the first one.
Start a discussion