0
量化版本
73
实测记录
量化版本
社区与官方发布的量化版本;「关联实测」只统计精确关联到该发布的实测记录。
还没有已收录的量化版本。
实测数据
共 50 组配置
| 硬件 | 框架 | 量化 | 生成速度 | 运行显存 | 实测次数 | 证据 |
|---|---|---|---|---|---|---|
| NVIDIA RTX 4090 | Ollama | Q3_K_M | 158.5 tok/s | 13.52 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4090 | Strata | Q3_K_M | 243.8 tok/s | 13.52 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 3090 | llama.cpp | IQ3_S | 121.9 tok/s | 13.19 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 3090 | vLLM | IQ4_XS | 102.6 tok/s | 14.53 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 7900 XTX | vLLM | IQ3_XS | 133.2 tok/s | 12.52 GB | 1 | L2 跨框架验证 |
| AMD Radeon RX 7900 XTX | Ollama | Q5_K_M | 71.3 tok/s | 17.72 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5080 | vLLM | IQ2_XS | 301.3 tok/s | 10.16 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5080 | Strata | IQ4_XS | 314.5 tok/s | 14.53 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 3060 12GB | vLLM | IQ3_XS | 89 tok/s | 11.4 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 3060 12GB | Ollama | Q3_K_S | 79.4 tok/s | 11.4 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 4080 Super | Ollama | Q2_K | 222.5 tok/s | 11.34 GB | 1 | L2 跨框架验证 |
| Apple M5 Ultra | llama.cpp | IQ3_S | 153.4 tok/s | 13.19 GB | 1 | L2 跨框架验证 |
| Apple M5 Ultra | Ollama | IQ2_M | 156.4 tok/s | 10.84 GB | 1 | L2 跨框架验证 |
| NVIDIA H100 80GB HBM3 | llama.cpp | IQ3_XS | 297.4 tok/s | 12.52 GB | 1 | L2 跨框架验证 |
| NVIDIA H100 80GB HBM3 | vLLM | Q3_K_S | 343.3 tok/s | 13.02 GB | 1 | L2 跨框架验证 |
| NVIDIA H20 | llama.cpp | IQ2_XS | 95.2 tok/s | 10.16 GB | 1 | L2 跨框架验证 |
| NVIDIA H20 | vLLM | Q3_K_M | 78.4 tok/s | 13.52 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX PRO 6000 Blackwell 96GB | vLLM | IQ3_XS | 344.9 tok/s | 12.52 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX PRO 6000 Blackwell 96GB | Ollama | IQ2_XS | 387.7 tok/s | 10.16 GB | 1 | L2 跨框架验证 |
| Intel Arc B580 | llama.cpp | IQ3_XS | 70.8 tok/s | 11.4 GB | 1 | L2 跨框架验证 |
| Intel Arc B580 | Ollama | IQ3_S | 65.1 tok/s | 11.4 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX Spark | llama.cpp | IQ3_S | 105.3 tok/s | 13.19 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX Spark | Ollama | IQ2_M | 116.7 tok/s | 10.84 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | vLLM | Q3_K_L | 336.5 tok/s | 8.15 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | vLLM | Q3_K_S | 310.9 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Strata | Q3_K_S | 326.1 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| NVIDIA RTX 5090 | Strata | Q5_K_M | 361.3 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | vLLM | IQ4_XS | 142.3 tok/s | 8.32 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | vLLM | Q3_K_S | 163.7 tok/s | 7.56 GB | 1 | L2 跨框架验证 |
| AMD Ryzen AI Max+ 395 | Strata | IQ2_XS | 273.7 tok/s | 6.13 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Ollama | IQ3_XS | 264.7 tok/s | 7.31 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Ollama | Q2_K | 303 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Strata | FP16 | 155.1 tok/s | 17.64 GB | 1 | L2 跨框架验证 |
| Apple M5 Max | Strata | Q3_K_L | 374.8 tok/s | 8.15 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | llama.cpp | Q2_K | 258.2 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| NVIDIA DGX Spark | vLLM | FP16 | 107.3 tok/s | 17.64 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | vLLM | IQ3_S | 89.5 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | vLLM | Q8_0 | 46.8 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | IQ2_M | 83.5 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| NVIDIA Jetson AGX Orin 64GB | Ollama | IQ3_S | 82 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | llama.cpp | IQ2_M | 142.2 tok/s | 6.47 GB | 1 | L2 跨框架验证 |
| AMD Strix Halo (Radeon 8060S) | Strata | Q8_0 | 92.3 tok/s | 11.76 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | vLLM | IQ3_S | 197.3 tok/s | 7.64 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | vLLM | Q5_K_M | 149.2 tok/s | 9.91 GB | 1 | L2 跨框架验证 |
| 2x AMD Strix Halo | Strata | Q2_K | 311.8 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | IQ3_XS | 93.7 tok/s | 7.31 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | vLLM | Q6_K | 56.3 tok/s | 10.5 GB | 1 | L2 跨框架验证 |
| AMD Strix Point (Radeon 890M) | Ollama | Q2_K | 85.3 tok/s | 6.72 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | llama.cpp | Q3_K_M | 76.3 tok/s | 7.81 GB | 1 | L2 跨框架验证 |
| Intel Arc A770 16GB | llama.cpp | Q4_K_S | 63.5 tok/s | 8.65 GB | 1 | L2 跨框架验证 |
其中 73 条实测未关联到具体量化版本,完整数据见搜索页。
社区讨论
还没有讨论——发起第一个话题吧。
发起讨论