Apple M2 Ultra
Other107
Run records
33
Models covered
4
Frameworks covered
L1 Reproduced × 1 · L2 Cross-framework verified × 106
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · Q3_K_M Decode 119.5 · Prefill 580.12 ·
- Ollama · IQ3_S Decode 108.26 · Prefill 586.93 ·
- llama.cpp · Q2_K Decode 325.56 · Prefill 1312.17 ·
- Ollama · IQ4_XS Decode 260.88 · Prefill 1133.93 ·
- llama.cpp · Q4_K_M Decode 119.47 · Prefill 397.94 ·
- Ollama · Q4_K_M Decode 118.08 · Prefill 636.62 ·
- llama.cpp · Q5_K_S Decode 159.13 · Prefill 607.98 ·
- Ollama · IQ4_XS Decode 184.42 · Prefill 830.52 ·
- llama.cpp · IQ2_M Decode 155.94 · Prefill 578.17 ·
- llama.cpp · IQ3_S Decode 130 · Prefill 684.42 ·
- Ollama · Q4_K_M Decode 110.45 · Prefill 501.39 ·
- llama.cpp · IQ2_M Decode 102.29 · Prefill 417.96 ·
- llama.cpp · Q5_K_M Decode 67.37 · Prefill 314.26 ·
- Ollama · IQ1_S Decode 100.6 · Prefill 448.02 ·
- llama.cpp · Q4_K_M Decode 92.52 · Prefill 377.54 ·
- llama.cpp · Q4_K_S Decode 88.6 · Prefill 484.39 ·
- Ollama · Q6_K Decode 67.84 · Prefill 354.32 ·
- llama.cpp · IQ2_XS Decode 77.57 · Prefill 340.43 ·
- Ollama · IQ2_XS Decode 73.6 · Prefill 261.9 ·
- Ollama · IQ3_XS Decode 64.55 · Prefill 256.75 ·
- llama.cpp · Q4_K_M Decode 67.38 · Prefill 344.18 ·
- Ollama · Q3_K_L Decode 80.85 · Prefill 325.8 ·
- llama.cpp · IQ1_S Decode 250.48 · Prefill 981.93 ·
- llama.cpp · Q8_0 Decode 114.98 · Prefill 558.64 ·
- Ollama · Q2_K Decode 217.13 · Prefill 942.96 ·
- llama.cpp · IQ3_XS Decode 156.36 · Prefill 856.45 ·
- llama.cpp · Q3_K_L Decode 147.11 · Prefill 678.92 ·
- Ollama · IQ2_M Decode 176 · Prefill 698.29 ·
- llama.cpp · Q3_K_L Decode 82.49 · Prefill 401.83 ·
- Ollama · IQ2_M Decode 104.98 · Prefill 551.16 ·
- llama.cpp · IQ1_S Decode 344.97 · Prefill 1343.81 ·
- llama.cpp · Q6_K Decode 188.49 · Prefill 853.9 ·
- Ollama · FP16 Decode 117 · Prefill 552.38 ·
- Ollama · Q8_0 Decode 150.72 · Prefill 712.42 ·
- llama.cpp · Q3_K_L Decode 110.22 · Prefill 411.86 ·
- Ollama · IQ3_XS Decode 108.03 · Prefill 420.83 ·
- llama.cpp · Q3_K_S Decode 119.1 · Prefill 434.44 ·
- Ollama · Q2_K Decode 112.25 · Prefill 576.42 ·
- llama.cpp · Q4_K_M Decode 58.99 · Prefill 259.24 ·
- Ollama · IQ2_XS Decode 73.22 · Prefill 389.92 ·
- llama.cpp · IQ4_XS Decode 90.66 · Prefill 438.77 ·
- llama.cpp · Q5_K_S Decode 66.9 · Prefill 326.75 ·
- Ollama · Q2_K Decode 96.52 · Prefill 478.87 ·
- llama.cpp · Q3_K_L Decode 129.48 · Prefill 476.49 ·
- llama.cpp · Q4_K_S Decode 123.15 · Prefill 540.31 ·
- Ollama · Q4_K_S Decode 103.18 · Prefill 542.74 ·
- llama.cpp · IQ2_M Decode 144.03 · Prefill 634.04 ·
- llama.cpp · Q2_K Decode 151.53 · Prefill 649.5 ·
- Ollama · IQ3_S Decode 116.94 · Prefill 440.9 ·
- Ollama · Q3_K_M Decode 114.21 · Prefill 572.36 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
VRAM is not recorded for this hardware; fit cannot be computed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- Apple
- Memory bandwidth
- 800 GB/s
Run records
50 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Ministral-3-8B-Instruct-2512 | llama.cpp | Q3_K_M | 119.5 tok/s | — | 1 |
| Ministral-3-8B-Instruct-2512 | Ollama | IQ3_S | 108.3 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | llama.cpp | Q2_K | 325.6 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | Ollama | IQ4_XS | 260.9 tok/s | — | 1 |
| NVIDIA-Nemotron-3.5-Bolt-E4B-4GB | llama.cpp | Q4_K_M | 119.5 tok/s | — | 1 |
| NVIDIA-Nemotron-3.5-Bolt-E4B-4GB | Ollama | Q4_K_M | 118.1 tok/s | — | 1 |
| Flint-40B-A2.5B | llama.cpp | Q5_K_S | 159.1 tok/s | — | 1 |
| Flint-40B-A2.5B | Ollama | IQ4_XS | 184.4 tok/s | — | 1 |
| Raon-1-mini-preview | llama.cpp | IQ2_M | 155.9 tok/s | — | 1 |
| Raon-1-mini-preview | llama.cpp | IQ3_S | 130 tok/s | — | 1 |
| Raon-1-mini-preview | Ollama | Q4_K_M | 110.5 tok/s | — | 1 |
| mammoth-1 | llama.cpp | IQ2_M | 102.3 tok/s | — | 1 |
| mammoth-1 | llama.cpp | Q5_K_M | 67.4 tok/s | — | 1 |
| mammoth-1 | Ollama | IQ1_S | 100.6 tok/s | — | 1 |
| elephant-1 | llama.cpp | Q4_K_M | 92.5 tok/s | — | 1 |
| elephant-1 | llama.cpp | Q4_K_S | 88.6 tok/s | — | 1 |
| elephant-1 | Ollama | Q6_K | 67.8 tok/s | — | 1 |
| Qwen3.8-27B | llama.cpp | IQ2_XS | 77.6 tok/s | — | 1 |
| Qwen3.8-27B | Ollama | IQ2_XS | 73.6 tok/s | — | 1 |
| Qwen3.8-27B | Ollama | IQ3_XS | 64.5 tok/s | — | 1 |
| Qwen3.6-27B | llama.cpp | Q4_K_M | 67.4 tok/s | — | 1 |
| Qwen3.6-27B | Ollama | Q3_K_L | 80.8 tok/s | — | 1 |
| Qwen3.6-35B-A3B | llama.cpp | IQ1_S | 250.5 tok/s | — | 1 |
| Qwen3.6-35B-A3B | llama.cpp | Q8_0 | 115 tok/s | — | 1 |
| Qwen3.6-35B-A3B | Ollama | Q2_K | 217.1 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | IQ3_XS | 156.4 tok/s | — | 1 |
| Qwen3.5-4B | llama.cpp | Q3_K_L | 147.1 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | IQ2_M | 176 tok/s | — | 2 |
| Qwen3.5-27B | llama.cpp | Q3_K_L | 82.5 tok/s | — | 1 |
| Qwen3.5-27B | Ollama | IQ2_M | 105 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | IQ1_S | 345 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | Q6_K | 188.5 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | FP16 | 117 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | Q8_0 | 150.7 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | Q3_K_L | 110.2 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | IQ3_XS | 108 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q3_K_S | 119.1 tok/s | — | 1 |
| Qwen3-8B | Ollama | Q2_K | 112.3 tok/s | — | 1 |
| gemma-4-31B-it | llama.cpp | Q4_K_M | 59 tok/s | — | 1 |
| gemma-4-31B-it | Ollama | IQ2_XS | 73.2 tok/s | — | 1 |
| gemma-4-12B-it | llama.cpp | IQ4_XS | 90.7 tok/s | — | 1 |
| gemma-4-12B-it | llama.cpp | Q5_K_S | 66.9 tok/s | — | 1 |
| gemma-4-12B-it | Ollama | Q2_K | 96.5 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | Q3_K_L | 129.5 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | Q4_K_S | 123.2 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Ollama | Q4_K_S | 103.2 tok/s | — | 1 |
| Llama-3-8B-Instruct | llama.cpp | IQ2_M | 144 tok/s | — | 1 |
| Llama-3-8B-Instruct | llama.cpp | Q2_K | 151.5 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | IQ3_S | 116.9 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | Q3_K_M | 114.2 tok/s | — | 1 |