0
量化版本
67
实测记录
1
讨论数
量化版本
社区与官方发布的量化版本；「关联实测」只统计精确关联到该发布的实测记录。
还没有已收录的量化版本。
实测数据
共 50 组配置
| 硬件 | 框架 | 量化 | 生成速度 | 运行显存 | 实测次数 | 证据 |
|---|---|---|---|---|---|---|
| NVIDIA RTX 5090 | llama.cpp | Q2_K | 277.3 tok/s | 12.5 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Ollama | Q5_K_M | 166 tok/s | 19.68 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | vLLM | IQ3_XS | 115.9 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | vLLM | Q5_K_M | 83.9 tok/s | 19.68 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | Ollama | Q4_K_S | 85.5 tok/s | 16.84 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | vLLM | IQ3_S | 222.5 tok/s | 14.57 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | vLLM | Q5_K_S | 163.1 tok/s | 19.11 GB | 2 | L2 跨框架验证 |
| Apple M5 Max | Strata | Q3_K_M | 284.3 tok/s | 14.95 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | llama.cpp | Q6_K | 110.1 tok/s | 21 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | Strata | Q3_K_M | 209.3 tok/s | 14.95 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | Q3_K_S | 55.4 tok/s | 14.38 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Strata | Q4_K_M | 72 tok/s | 17.6 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | llama.cpp | IQ1_S | 116.2 tok/s | 10.04 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | llama.cpp | Q5_K_S | 60.5 tok/s | 19.11 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | Q4_K_S | 105.7 tok/s | 16.84 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | Q6_K | 73.1 tok/s | 21 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Ollama | FP16 | 58.4 tok/s | 37.06 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Strata | IQ3_XS | 234.6 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | IQ3_S | 65.9 tok/s | 14.57 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Ollama | IQ3_XS | 62.9 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Ollama | IQ4_XS | 46.8 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | IQ3_XS | 56.6 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | vLLM | Q3_K_L | 54.6 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | Ollama | IQ2_XS | 61.1 tok/s | 11.17 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | Q3_K_M | 51.9 tok/s | 14.95 GB | 1 | L2 跨框架验证 |
| Apple M1 | Strata | IQ1_S | 96.9 tok/s | 10.04 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | llama.cpp | IQ3_XS | 75.7 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | vLLM | IQ2_M | 84.2 tok/s | 11.93 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | vLLM | Q3_K_S | 79.3 tok/s | 14.38 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | llama.cpp | Q5_K_M | 65.4 tok/s | 19.68 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Strata | FP16 | 50.1 tok/s | 37.06 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Strata | Q5_K_M | 86.9 tok/s | 19.68 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Ollama | Q3_K_L | 72.5 tok/s | 15.71 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Ollama | Q5_K_M | 53 tok/s | 19.68 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Strata | Q3_K_M | 98.4 tok/s | 14.95 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Strata | Q5_K_S | 84.6 tok/s | 19.11 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | llama.cpp | Q3_K_S | 108.4 tok/s | 14.38 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | vLLM | FP16 | 54.5 tok/s | 37.06 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | vLLM | Q3_K_L | 115.9 tok/s | 15.71 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Strata | Q4_K_S | 60 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4060 Ti 16GB | Ollama | IQ3_S | 73.1 tok/s | 14.57 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 XT | vLLM | IQ1_S | 133.2 tok/s | 10.04 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 XT | vLLM | Q3_K_S | 113.7 tok/s | 14.38 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 XT | Ollama | IQ2_XS | 124.5 tok/s | 11.17 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 | Ollama | Q2_K | 96.6 tok/s | 12.5 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 | Ollama | Q4_K_S | 80.6 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 9070 | Strata | Q4_K_S | 115.5 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | llama.cpp | IQ3_XS | 118.3 tok/s | 13.82 GB | 1 | L2 跨框架验证 |
| Apple M4 Pro | Strata | IQ2_M | 177.4 tok/s | 11.93 GB | 1 | L2 跨框架验证 |
| Apple M5 Pro | vLLM | Q3_K_L | 145.9 tok/s | 15.71 GB | 1 | L2 跨框架验证 |
其中 67 条实测未关联到具体量化版本，完整数据见搜索页。