Qwen3-4B-Thinking-2507

4B推理由 Qwen 发布 发起讨论

0

量化版本

104

实测记录

量化版本

社区与官方发布的量化版本;「关联实测」只统计精确关联到该发布的实测记录。

还没有已收录的量化版本。

实测数据

共 50 组配置

硬件框架量化生成速度运行显存实测次数证据
NVIDIA RTX 5090llama.cppQ2_K392.5 tok/s6.72 GB1L2 跨框架验证
NVIDIA RTX 5090llama.cppQ8_0184.6 tok/s11.76 GB1L2 跨框架验证
NVIDIA RTX 5090vLLMIQ2_M386.9 tok/s6.47 GB1L2 跨框架验证
NVIDIA RTX 5090vLLMQ3_K_L337.2 tok/s8.15 GB1L2 跨框架验证
AMD Ryzen AI Max+ 395llama.cppQ5_K_M107.4 tok/s9.91 GB2L2 跨框架验证
AMD Ryzen AI Max+ 395llama.cppQ8_078.3 tok/s11.76 GB1L2 跨框架验证
AMD Ryzen AI Max+ 395OllamaQ3_K_S145 tok/s7.56 GB1L2 跨框架验证
Apple M5 MaxOllamaQ5_K_S192.3 tok/s9.66 GB1L2 跨框架验证
Apple M5 MaxStrataQ5_K_M281.2 tok/s9.91 GB1L2 跨框架验证
NVIDIA DGX SparkvLLMIQ1_S291.5 tok/s5.63 GB1L2 跨框架验证
NVIDIA DGX SparkvLLMQ5_K_M151.7 tok/s9.91 GB1L2 跨框架验证
NVIDIA DGX SparkStrataQ5_K_M220.6 tok/s9.91 GB1L2 跨框架验证
NVIDIA Jetson AGX Orin 64GBvLLMQ3_K_S89.8 tok/s7.56 GB1L2 跨框架验证
NVIDIA Jetson AGX Orin 64GBvLLMQ5_K_S57.1 tok/s9.66 GB1L2 跨框架验证
NVIDIA Jetson AGX Orin 64GBOllamaQ2_K77 tok/s6.72 GB1L2 跨框架验证
AMD Strix Halo (Radeon 8060S)llama.cppQ3_K_S117 tok/s7.56 GB1L2 跨框架验证
AMD Strix Halo (Radeon 8060S)llama.cppQ5_K_M78 tok/s9.91 GB1L2 跨框架验证
AMD Strix Halo (Radeon 8060S)StrataIQ2_XS206.2 tok/s6.13 GB1L2 跨框架验证
AMD Strix Halo (Radeon 8060S)StrataIQ3_S166.3 tok/s7.64 GB1L2 跨框架验证
2x AMD Strix Halollama.cppQ3_K_M181.3 tok/s7.81 GB1L2 跨框架验证
2x AMD Strix Halollama.cppQ4_K_M153.9 tok/s8.99 GB1L2 跨框架验证
2x AMD Strix HaloStrataIQ2_M312.1 tok/s6.47 GB1L2 跨框架验证
AMD Strix Point (Radeon 890M)llama.cppIQ4_XS75.7 tok/s8.32 GB1L2 跨框架验证
AMD Strix Point (Radeon 890M)vLLMQ5_K_S60.1 tok/s9.66 GB1L2 跨框架验证
Intel Arc A770 16GBOllamaIQ3_XS72.9 tok/s7.31 GB2L2 跨框架验证
Intel Arc A770 16GBOllamaQ8_038.6 tok/s11.76 GB1L2 跨框架验证
Intel Arc A770 16GBStrataIQ3_S110.5 tok/s7.64 GB1L2 跨框架验证
Intel Arc 140VvLLMQ3_K_M77.5 tok/s7.81 GB1L2 跨框架验证
Intel Arc 140VOllamaQ4_K_S56.9 tok/s8.65 GB1L2 跨框架验证
Apple M1llama.cppIQ2_XS75.2 tok/s6.13 GB1L2 跨框架验证
Apple M1llama.cppQ4_K_M59.4 tok/s8.99 GB1L2 跨框架验证
Apple M1vLLMFP1630.1 tok/s15.2 GB1L2 跨框架验证
Apple M1vLLMIQ2_XS79.9 tok/s6.13 GB1L2 跨框架验证
Apple M1 ProvLLMIQ3_S94.5 tok/s7.64 GB1L2 跨框架验证
Apple M1 ProStrataIQ1_S163.5 tok/s5.63 GB1L2 跨框架验证
Apple M1 ProStrataQ2_K151.4 tok/s6.72 GB1L2 跨框架验证
Apple M1 MaxvLLMFP1648.3 tok/s17.64 GB1L2 跨框架验证
Apple M1 MaxStrataIQ3_XS182.5 tok/s7.31 GB1L2 跨框架验证
Apple M1 MaxStrataQ4_K_M125 tok/s8.99 GB1L2 跨框架验证
Apple M2 ProvLLMIQ3_S112.3 tok/s7.64 GB1L2 跨框架验证
Apple M2 ProStrataIQ2_M171.6 tok/s6.47 GB1L2 跨框架验证
Apple M2 ProStrataQ4_K_S119.2 tok/s8.65 GB1L2 跨框架验证
Apple M2 UltravLLMIQ2_M191.2 tok/s6.47 GB1L2 跨框架验证
Apple M2 UltraOllamaQ3_K_M131.4 tok/s7.81 GB1L2 跨框架验证
Qualcomm Snapdragon X Elitellama.cppQ2_K81.6 tok/s6.72 GB1L2 跨框架验证
Qualcomm Snapdragon X EliteStrataQ8_048.7 tok/s11.76 GB1L2 跨框架验证
Qualcomm Snapdragon 7 Gen 4llama.cppQ3_K_S53.5 tok/s7.56 GB1L2 跨框架验证
NVIDIA RTX 4060 Ti 16GBvLLMIQ3_S108.2 tok/s7.64 GB1L2 跨框架验证
NVIDIA RTX 4060 Ti 16GBvLLMQ3_K_M119.5 tok/s7.81 GB1L2 跨框架验证
NVIDIA RTX 4060 Ti 16GBStrataIQ3_XS160.6 tok/s7.31 GB1L2 跨框架验证

其中 104 条实测未关联到具体量化版本,完整数据见搜索页。

社区讨论

还没有讨论——发起第一个话题吧。

发起讨论