0
Quantized builds
5
Benchmark records
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
5 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| Apple M3 Max | llama.cpp | IQ2_XS | 89.4 tok/s | 34.2 GB | 1 | L2 Cross-framework verified |
| Apple M3 Max | vLLM | IQ1_S | 102.6 tok/s | 32.97 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | llama.cpp | IQ3_XS | 121.6 tok/s | 47.67 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | llama.cpp | Q4_K_S | 111 tok/s | 59.43 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | Ollama | Q3_K_L | 102.3 tok/s | 55.02 GB | 1 | L2 Cross-framework verified |
5 of these records are not linked to a specific quantized build; the full set is on the search page.
Community discussion
No discussions yet — start the first one.
Start a discussion