Qwen3-4B-Thinking-2507

4BReasoningby Qwen Start a discussion

0

Quantized builds

104

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

50 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX 5090llama.cppQ2_K392.5 tok/s6.72 GB1L2 Cross-framework verified
NVIDIA RTX 5090llama.cppQ8_0184.6 tok/s11.76 GB1L2 Cross-framework verified
NVIDIA RTX 5090vLLMIQ2_M386.9 tok/s6.47 GB1L2 Cross-framework verified
NVIDIA RTX 5090vLLMQ3_K_L337.2 tok/s8.15 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppQ5_K_M107.4 tok/s9.91 GB2L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppQ8_078.3 tok/s11.76 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaQ3_K_S145 tok/s7.56 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ5_K_S192.3 tok/s9.66 GB1L2 Cross-framework verified
Apple M5 MaxStrataQ5_K_M281.2 tok/s9.91 GB1L2 Cross-framework verified
NVIDIA DGX SparkvLLMIQ1_S291.5 tok/s5.63 GB1L2 Cross-framework verified
NVIDIA DGX SparkvLLMQ5_K_M151.7 tok/s9.91 GB1L2 Cross-framework verified
NVIDIA DGX SparkStrataQ5_K_M220.6 tok/s9.91 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMQ3_K_S89.8 tok/s7.56 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMQ5_K_S57.1 tok/s9.66 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBOllamaQ2_K77 tok/s6.72 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)llama.cppQ3_K_S117 tok/s7.56 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)llama.cppQ5_K_M78 tok/s9.91 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)StrataIQ2_XS206.2 tok/s6.13 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)StrataIQ3_S166.3 tok/s7.64 GB1L2 Cross-framework verified
2x AMD Strix Halollama.cppQ3_K_M181.3 tok/s7.81 GB1L2 Cross-framework verified
2x AMD Strix Halollama.cppQ4_K_M153.9 tok/s8.99 GB1L2 Cross-framework verified
2x AMD Strix HaloStrataIQ2_M312.1 tok/s6.47 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)llama.cppIQ4_XS75.7 tok/s8.32 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)vLLMQ5_K_S60.1 tok/s9.66 GB1L2 Cross-framework verified
Intel Arc A770 16GBOllamaIQ3_XS72.9 tok/s7.31 GB2L2 Cross-framework verified
Intel Arc A770 16GBOllamaQ8_038.6 tok/s11.76 GB1L2 Cross-framework verified
Intel Arc A770 16GBStrataIQ3_S110.5 tok/s7.64 GB1L2 Cross-framework verified
Intel Arc 140VvLLMQ3_K_M77.5 tok/s7.81 GB1L2 Cross-framework verified
Intel Arc 140VOllamaQ4_K_S56.9 tok/s8.65 GB1L2 Cross-framework verified
Apple M1llama.cppIQ2_XS75.2 tok/s6.13 GB1L2 Cross-framework verified
Apple M1llama.cppQ4_K_M59.4 tok/s8.99 GB1L2 Cross-framework verified
Apple M1vLLMFP1630.1 tok/s15.2 GB1L2 Cross-framework verified
Apple M1vLLMIQ2_XS79.9 tok/s6.13 GB1L2 Cross-framework verified
Apple M1 ProvLLMIQ3_S94.5 tok/s7.64 GB1L2 Cross-framework verified
Apple M1 ProStrataIQ1_S163.5 tok/s5.63 GB1L2 Cross-framework verified
Apple M1 ProStrataQ2_K151.4 tok/s6.72 GB1L2 Cross-framework verified
Apple M1 MaxvLLMFP1648.3 tok/s17.64 GB1L2 Cross-framework verified
Apple M1 MaxStrataIQ3_XS182.5 tok/s7.31 GB1L2 Cross-framework verified
Apple M1 MaxStrataQ4_K_M125 tok/s8.99 GB1L2 Cross-framework verified
Apple M2 ProvLLMIQ3_S112.3 tok/s7.64 GB1L2 Cross-framework verified
Apple M2 ProStrataIQ2_M171.6 tok/s6.47 GB1L2 Cross-framework verified
Apple M2 ProStrataQ4_K_S119.2 tok/s8.65 GB1L2 Cross-framework verified
Apple M2 UltravLLMIQ2_M191.2 tok/s6.47 GB1L2 Cross-framework verified
Apple M2 UltraOllamaQ3_K_M131.4 tok/s7.81 GB1L2 Cross-framework verified
Qualcomm Snapdragon X Elitellama.cppQ2_K81.6 tok/s6.72 GB1L2 Cross-framework verified
Qualcomm Snapdragon X EliteStrataQ8_048.7 tok/s11.76 GB1L2 Cross-framework verified
Qualcomm Snapdragon 7 Gen 4llama.cppQ3_K_S53.5 tok/s7.56 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBvLLMIQ3_S108.2 tok/s7.64 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBvLLMQ3_K_M119.5 tok/s7.81 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBStrataIQ3_XS160.6 tok/s7.31 GB1L2 Cross-framework verified

104 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion