0
量化版本
104
实测记录
量化版本
社区与官方发布的量化版本;「关联实测」只统计精确关联到该发布的实测记录。
还没有已收录的量化版本。
实测数据
共 50 组配置
| 硬件 | 框架 | 量化 | 生成速度 | 运行显存 | 实测次数 | 证据 |
|---|---|---|---|---|---|---|
| NVIDIA RTX 5090 | llama.cpp | Q2_K | 392.5 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | llama.cpp | Q8_0 | 184.6 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | vLLM | IQ2_M | 386.9 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | vLLM | Q3_K_L | 337.2 tok/s | 8.15 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | llama.cpp | Q5_K_M | 107.4 tok/s | 9.91 GB | 2 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | llama.cpp | Q8_0 | 78.3 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | Ollama | Q3_K_S | 145 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Ollama | Q5_K_S | 192.3 tok/s | 9.66 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Strata | Q5_K_M | 281.2 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | IQ1_S | 291.5 tok/s | 5.63 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | Q5_K_M | 151.7 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | Strata | Q5_K_M | 220.6 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | vLLM | Q3_K_S | 89.8 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | vLLM | Q5_K_S | 57.1 tok/s | 9.66 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | Q2_K | 77 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | llama.cpp | Q3_K_S | 117 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | llama.cpp | Q5_K_M | 78 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | IQ2_XS | 206.2 tok/s | 6.13 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | IQ3_S | 166.3 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | llama.cpp | Q3_K_M | 181.3 tok/s | 7.81 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | llama.cpp | Q4_K_M | 153.9 tok/s | 8.99 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Strata | IQ2_M | 312.1 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | llama.cpp | IQ4_XS | 75.7 tok/s | 8.32 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | Q5_K_S | 60.1 tok/s | 9.66 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | IQ3_XS | 72.9 tok/s | 7.31 GB | 2 | L2 跨框架验证 |
| Intel Arc A770 16GB | Ollama | Q8_0 | 38.6 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | Strata | IQ3_S | 110.5 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | vLLM | Q3_K_M | 77.5 tok/s | 7.81 GB | 1 | L2 跨框架验证 |
| Intel Arc 140V | Ollama | Q4_K_S | 56.9 tok/s | 8.65 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | IQ2_XS | 75.2 tok/s | 6.13 GB | 1 | L2 跨框架验证 |
| Apple M1 | llama.cpp | Q4_K_M | 59.4 tok/s | 8.99 GB | 1 | L2 跨框架验证 |
| Apple M1 | vLLM | FP16 | 30.1 tok/s | 15.2 GB | 1 | L2 跨框架验证 |
| Apple M1 | vLLM | IQ2_XS | 79.9 tok/s | 6.13 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | vLLM | IQ3_S | 94.5 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Strata | IQ1_S | 163.5 tok/s | 5.63 GB | 1 | L2 跨框架验证 |
| Apple M1 Pro | Strata | Q2_K | 151.4 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | vLLM | FP16 | 48.3 tok/s | 17.64 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Strata | IQ3_XS | 182.5 tok/s | 7.31 GB | 1 | L2 跨框架验证 |
| Apple M1 Max | Strata | Q4_K_M | 125 tok/s | 8.99 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | vLLM | IQ3_S | 112.3 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Strata | IQ2_M | 171.6 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| Apple M2 Pro | Strata | Q4_K_S | 119.2 tok/s | 8.65 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | vLLM | IQ2_M | 191.2 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| Apple M2 Ultra | Ollama | Q3_K_M | 131.4 tok/s | 7.81 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | llama.cpp | Q2_K | 81.6 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon X Elite | Strata | Q8_0 | 48.7 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| Qualcomm Snapdragon 7 Gen 4 | llama.cpp | Q3_K_S | 53.5 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4060 Ti 16GB | vLLM | IQ3_S | 108.2 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4060 Ti 16GB | vLLM | Q3_K_M | 119.5 tok/s | 7.81 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4060 Ti 16GB | Strata | IQ3_XS | 160.6 tok/s | 7.31 GB | 1 | L2 跨框架验证 |
其中 104 条实测未关联到具体量化版本,完整数据见搜索页。
社区讨论
还没有讨论——发起第一个话题吧。
发起讨论