0
量化版本
53
实测记录
1
讨论数
量化版本
社区与官方发布的量化版本;「关联实测」只统计精确关联到该发布的实测记录。
还没有已收录的量化版本。
实测数据
共 50 组配置
| 硬件 | 框架 | 量化 | 生成速度 | 运行显存 | 实测次数 | 证据 |
|---|---|---|---|---|---|---|
| NVIDIA RTX 5090 | llama.cpp | Q3_K_L | 324.4 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Strata | IQ3_S | 332.3 tok/s | 4.87 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | llama.cpp | Q3_K_M | 208.8 tok/s | 4.96 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | vLLM | IQ1_S | 291.6 tok/s | 3.86 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | vLLM | Q3_K_M | 285.3 tok/s | 4.96 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Strata | Q3_K_L | 329.6 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Strata | Q8_0 | 323.1 tok/s | 6.93 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | IQ2_M | 380.9 tok/s | 4.28 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | IQ3_XS | 360 tok/s | 4.7 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | Strata | Q3_K_L | 389.3 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | Strata | Q4_K_M | 326.7 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | Q6_K | 70.7 tok/s | 6.3 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Strata | IQ2_M | 176.2 tok/s | 4.28 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Strata | IQ3_S | 167.9 tok/s | 4.87 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Ollama | FP16 | 70.6 tok/s | 9.87 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | Q3_K_S | 228.5 tok/s | 4.83 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | Q6_K | 157.8 tok/s | 6.3 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | vLLM | Q3_K_S | 277.8 tok/s | 4.83 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Strata | IQ3_XS | 321.6 tok/s | 4.7 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | llama.cpp | IQ1_S | 145.7 tok/s | 3.86 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | llama.cpp | IQ4_XS | 106 tok/s | 5.21 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Ollama | FP16 | 47.3 tok/s | 9.87 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Ollama | Q8_0 | 57.3 tok/s | 6.93 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | Q6_K | 63.9 tok/s | 6.3 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Strata | Q4_K_M | 120.1 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | llama.cpp | Q4_K_M | 78.9 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | llama.cpp | Q8_0 | 53.1 tok/s | 6.93 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | vLLM | Q4_K_M | 93.7 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | IQ2_XS | 125.4 tok/s | 4.12 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | Q6_K | 62.5 tok/s | 6.3 GB | 1 | L2 跨框架验证 |
| Apple M1 | Ollama | IQ2_XS | 113.2 tok/s | 4.12 GB | 1 | L2 跨框架验证 |
| Apple M1 | Ollama | IQ3_S | 89.3 tok/s | 4.87 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | llama.cpp | Q3_K_M | 141.1 tok/s | 4.96 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | llama.cpp | Q6_K | 93.8 tok/s | 6.3 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Ollama | Q3_K_L | 126.4 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Ollama | Q5_K_M | 87.8 tok/s | 6.01 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | vLLM | Q3_K_S | 189.6 tok/s | 4.83 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | vLLM | Q8_0 | 98.8 tok/s | 6.93 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Strata | IQ1_S | 324.3 tok/s | 3.86 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | vLLM | Q4_K_M | 128.4 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Ollama | IQ2_XS | 158 tok/s | 4.12 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Ollama | Q3_K_S | 140 tok/s | 4.83 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | llama.cpp | IQ2_XS | 255.2 tok/s | 4.12 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | llama.cpp | Q3_K_S | 222.2 tok/s | 4.83 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | Strata | Q3_K_L | 291.9 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Ollama | Q4_K_M | 76.5 tok/s | 5.54 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Strata | Q2_K | 154.6 tok/s | 4.41 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Strata | Q3_K_L | 121.8 tok/s | 5.12 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | vLLM | Q3_K_M | 80.6 tok/s | 4.96 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | vLLM | Q8_0 | 46.5 tok/s | 6.93 GB | 1 | L2 跨框架验证 |
其中 53 条实测未关联到具体量化版本,完整数据见搜索页。