0
量化版本
83
实测记录
量化版本
社区与官方发布的量化版本；「关联实测」只统计精确关联到该发布的实测记录。
还没有已收录的量化版本。
实测数据
共 50 组配置
| 硬件 | 框架 | 量化 | 生成速度 | 运行显存 | 实测次数 | 证据 |
|---|---|---|---|---|---|---|
| NVIDIA RTX 5090 | vLLM | IQ2_XS | 304.5 tok/s | 2.71 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | vLLM | Q3_K_S | 322.2 tok/s | 2.92 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Strata | Q2_K | 350.6 tok/s | 2.79 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Strata | Q5_K_M | 291.9 tok/s | 3.28 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | llama.cpp | IQ1_S | 285.6 tok/s | 2.62 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | Ollama | Q5_K_S | 220.1 tok/s | 3.23 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | llama.cpp | Q6_K | 297.4 tok/s | 3.36 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | vLLM | FP16 | 257.7 tok/s | 4.43 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | vLLM | IQ2_M | 302.9 tok/s | 2.75 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | IQ1_S | 353.2 tok/s | 2.62 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | Q5_K_S | 364.7 tok/s | 3.23 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | Ollama | IQ2_M | 338.9 tok/s | 2.75 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | llama.cpp | IQ1_S | 232.9 tok/s | 2.62 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | llama.cpp | Q5_K_S | 140.4 tok/s | 3.23 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | Q5_K_S | 120.5 tok/s | 3.23 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | vLLM | IQ2_XS | 355.9 tok/s | 2.71 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | vLLM | IQ3_S | 267 tok/s | 2.93 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | IQ3_S | 374.2 tok/s | 2.93 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Ollama | FP16 | 181.9 tok/s | 4.43 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Ollama | IQ4_XS | 379.8 tok/s | 3.03 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Strata | IQ4_XS | 288.7 tok/s | 3.03 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | Q4_K_S | 178.1 tok/s | 3.09 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | Q8_0 | 112.4 tok/s | 3.55 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Strata | IQ2_M | 281.1 tok/s | 2.75 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Strata | IQ3_XS | 273.4 tok/s | 2.88 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | vLLM | Q4_K_S | 160.5 tok/s | 3.09 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | IQ4_XS | 153.4 tok/s | 3.03 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | Q5_K_M | 116 tok/s | 3.28 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | vLLM | IQ2_M | 187.5 tok/s | 2.75 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | vLLM | IQ3_S | 170 tok/s | 2.93 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | Ollama | IQ3_S | 157.2 tok/s | 2.93 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | IQ4_XS | 148.8 tok/s | 3.03 GB | 1 | L2 跨框架验证 |
| Apple M1 | Ollama | Q2_K | 166.6 tok/s | 2.79 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | vLLM | Q5_K_M | 164.4 tok/s | 3.28 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Ollama | IQ1_S | 247.7 tok/s | 2.62 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Ollama | Q4_K_S | 189.9 tok/s | 3.09 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | vLLM | Q3_K_M | 253.6 tok/s | 2.96 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Ollama | Q2_K | 294.7 tok/s | 2.79 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Ollama | Q3_K_L | 236.5 tok/s | 3 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | vLLM | Q5_K_M | 162.8 tok/s | 3.28 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Ollama | Q5_K_M | 141.8 tok/s | 3.28 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | vLLM | FP16 | 152.7 tok/s | 4.43 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | vLLM | Q2_K | 308 tok/s | 2.79 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | Strata | IQ3_XS | 284.9 tok/s | 2.88 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | vLLM | Q4_K_S | 152.6 tok/s | 3.09 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Strata | Q6_K | 148.3 tok/s | 3.36 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | llama.cpp | IQ2_XS | 156.1 tok/s | 2.71 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | Strata | Q3_K_M | 155.2 tok/s | 2.96 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | Strata | Q5_K_S | 117.8 tok/s | 3.23 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4060 Ti 16GB | llama.cpp | Q4_K_S | 221.6 tok/s | 3.09 GB | 1 | L2 跨框架验证 |
其中 83 条实测未关联到具体量化版本，完整数据见搜索页。
社区讨论
还没有讨论——发起第一个话题吧。
发起讨论