Apple M1
Other39
Run records
14
Models covered
4
Frameworks covered
L2 Cross-framework verified × 39
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ2_M Decode 88.05 · Prefill 311.52 ·
- llama.cpp · Q4_K_M Decode 55.57 · Prefill 194.52 ·
- Strata · Q4_K_S Decode 87.2 · Prefill 469.35 ·
- llama.cpp · Q3_K_M Decode 51.89 · Prefill 170.87 ·
- Strata · IQ1_S Decode 96.89 · Prefill 527.4 ·
- vLLM · Q4_K_S Decode 132.87 · Prefill 708.61 ·
- Strata · Q2_K Decode 231.36 · Prefill 1029.89 ·
- Ollama · IQ3_XS Decode 50.37 · Prefill 163.76 ·
- Ollama · Q4_K_M Decode 39.07 · Prefill 131.6 ·
- Ollama · IQ2_XS Decode 64.59 · Prefill 227.22 ·
- llama.cpp · Q3_K_L Decode 65.56 · Prefill 344.27 ·
- llama.cpp · Q8_0 Decode 36.81 · Prefill 167.78 ·
- Ollama · Q2_K Decode 72.5 · Prefill 393.97 ·
- llama.cpp · IQ3_S Decode 97.72 · Prefill 393.93 ·
- llama.cpp · Q5_K_M Decode 75.84 · Prefill 355.79 ·
- vLLM · Q4_K_M Decode 84.3 · Prefill 383.2 ·
- vLLM · Q6_K Decode 72.53 · Prefill 321.64 ·
- llama.cpp · IQ2_XS Decode 75.23 · Prefill 381.59 ·
- llama.cpp · Q4_K_M Decode 59.36 · Prefill 212.58 ·
- vLLM · FP16 Decode 30.09 · Prefill 100.41 ·
- vLLM · IQ2_XS Decode 79.87 · Prefill 436.11 ·
- vLLM · IQ4_XS Decode 47.87 · Prefill 181.96 ·
- Ollama · FP16 Decode 21.35 · Prefill 72.87 ·
- Ollama · Q3_K_L Decode 41.49 · Prefill 134.08 ·
- llama.cpp · IQ2_M Decode 189.83 · Prefill 1001.05 ·
- llama.cpp · Q2_K Decode 189.43 · Prefill 860.24 ·
- vLLM · Q3_K_S Decode 170.33 · Prefill 911.66 ·
- vLLM · Q4_K_M Decode 143.97 · Prefill 610.72 ·
- Ollama · IQ3_XS Decode 74.16 · Prefill 283.84 ·
- Ollama · Q6_K Decode 48.37 · Prefill 198.04 ·
- Strata · Q3_K_M Decode 114.61 · Prefill 400.65 ·
- llama.cpp · IQ4_XS Decode 148.84 · Prefill 610.28 ·
- Ollama · Q2_K Decode 166.63 · Prefill 805.73 ·
- llama.cpp · IQ2_XS Decode 125.44 · Prefill 495.34 ·
- llama.cpp · Q6_K Decode 62.45 · Prefill 267.16 ·
- Ollama · IQ2_XS Decode 113.15 · Prefill 569.16 ·
- Ollama · IQ3_S Decode 89.33 · Prefill 357.04 ·
- Ollama · Q2_K Decode 70.49 · Prefill 362.07 ·
- Strata · Q5_K_M Decode 70.01 · Prefill 293.99 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
VRAM is not recorded for this hardware; fit cannot be computed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- Apple
- Memory bandwidth
- 68 GB/s
Run records
39 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-4B | llama.cpp | IQ2_M | 88 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | Q4_K_M | 55.6 tok/s | — | 1 |
| Qwen3.5-4B | Strata | Q4_K_S | 87.2 tok/s | — | 1 |
| Qwen3.5-9B | llama.cpp | Q3_K_M | 51.9 tok/s | — | 1 |
| Qwen3.5-9B | Strata | IQ1_S | 96.9 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | Q4_K_S | 132.9 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | Q2_K | 231.4 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | IQ3_XS | 50.4 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q4_K_M | 39.1 tok/s | — | 1 |
| Qwen3-8B | Ollama | IQ2_XS | 64.6 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q3_K_L | 65.6 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q8_0 | 36.8 tok/s | — | 1 |
| Qwen3-4B | Ollama | Q2_K | 72.5 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | IQ3_S | 97.7 tok/s | — | 1 |
| Qwen3-1.7B | llama.cpp | Q5_K_M | 75.8 tok/s | — | 1 |
| Qwen3-1.7B | vLLM | Q4_K_M | 84.3 tok/s | — | 1 |
| Qwen3-1.7B | vLLM | Q6_K | 72.5 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | llama.cpp | IQ2_XS | 75.2 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | llama.cpp | Q4_K_M | 59.4 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | FP16 | 30.1 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | vLLM | IQ2_XS | 79.9 tok/s | — | 1 |
| Qwen3-Coder-Next | vLLM | IQ4_XS | 47.9 tok/s | — | 1 |
| Qwen3-Coder-Next | Ollama | FP16 | 21.4 tok/s | — | 1 |
| Qwen3-Coder-Next | Ollama | Q3_K_L | 41.5 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | llama.cpp | IQ2_M | 189.8 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | llama.cpp | Q2_K | 189.4 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | vLLM | Q3_K_S | 170.3 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | vLLM | Q4_K_M | 144 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Ollama | IQ3_XS | 74.2 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Ollama | Q6_K | 48.4 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Strata | Q3_K_M | 114.6 tok/s | — | 1 |
| Qwen3-Embedding-0.6B | llama.cpp | IQ4_XS | 148.8 tok/s | — | 1 |
| Qwen3-Embedding-0.6B | Ollama | Q2_K | 166.6 tok/s | — | 1 |
| gemma-4-E2B-it | llama.cpp | IQ2_XS | 125.4 tok/s | — | 1 |
| gemma-4-E2B-it | llama.cpp | Q6_K | 62.5 tok/s | — | 1 |
| gemma-4-E2B-it | Ollama | IQ2_XS | 113.2 tok/s | — | 1 |
| gemma-4-E2B-it | Ollama | IQ3_S | 89.3 tok/s | — | 1 |
| gemma-4-E4B-it | Ollama | Q2_K | 70.5 tok/s | — | 1 |
| gemma-4-E4B-it | Strata | Q5_K_M | 70 tok/s | — | 1 |