Qwen3-4B

4BChatby Qwen Start a discussion

0

Quantized builds

90

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

50 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX 5090llama.cppFP16149.3 tok/s17.64 GB1L2 Cross-framework verified
NVIDIA RTX 5090llama.cppIQ4_XS335.8 tok/s8.32 GB1L2 Cross-framework verified
NVIDIA RTX 5090OllamaIQ3_XS349.8 tok/s7.31 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppQ6_K104.5 tok/s10.5 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaIQ1_S170.9 tok/s5.63 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaIQ2_M170.8 tok/s6.47 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppQ3_K_L253.6 tok/s8.15 GB1L2 Cross-framework verified
Apple M5 MaxOllamaIQ2_XS308.4 tok/s6.13 GB2L2 Cross-framework verified
Apple M5 MaxOllamaIQ3_XS249.1 tok/s7.31 GB1L2 Cross-framework verified
NVIDIA DGX Sparkllama.cppQ3_K_M206.5 tok/s7.81 GB1L2 Cross-framework verified
NVIDIA DGX SparkStrataQ3_K_S338 tok/s7.56 GB1L2 Cross-framework verified
NVIDIA DGX SparkStrataQ5_K_M221 tok/s9.91 GB2L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBllama.cppIQ2_M92.8 tok/s6.47 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMIQ2_M96.3 tok/s6.47 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMQ2_K95.8 tok/s6.72 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)llama.cppQ4_K_S106.7 tok/s8.65 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)StrataQ4_K_S150.6 tok/s8.65 GB1L2 Cross-framework verified
AMD Strix Halo (Radeon 8060S)StrataQ6_K108.8 tok/s10.5 GB1L2 Cross-framework verified
2x AMD Strix Halollama.cppFP1687 tok/s17.64 GB1L2 Cross-framework verified
2x AMD Strix HaloOllamaIQ3_S197.2 tok/s7.64 GB1L2 Cross-framework verified
2x AMD Strix HaloOllamaQ4_K_M165.7 tok/s8.99 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)vLLMFP1636.1 tok/s15.2 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)vLLMIQ2_XS112.8 tok/s6.13 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)OllamaIQ4_XS68.6 tok/s8.32 GB1L2 Cross-framework verified
AMD Strix Point (Radeon 890M)OllamaQ3_K_L75.1 tok/s8.15 GB1L2 Cross-framework verified
Intel Arc A770 16GBvLLMIQ4_XS72 tok/s8.32 GB1L2 Cross-framework verified
Intel Arc A770 16GBStrataQ6_K68.9 tok/s10.5 GB1L2 Cross-framework verified
Intel Arc 140VvLLMIQ3_S80.1 tok/s7.64 GB1L2 Cross-framework verified
Intel Arc 140VvLLMQ8_042.9 tok/s11.76 GB1L2 Cross-framework verified
Intel Arc 140VOllamaIQ1_S81.3 tok/s5.63 GB1L2 Cross-framework verified
Intel Arc 140VOllamaQ4_K_M54.5 tok/s8.99 GB1L2 Cross-framework verified
Apple M1llama.cppQ3_K_L65.6 tok/s8.15 GB1L2 Cross-framework verified
Apple M1llama.cppQ8_036.8 tok/s11.76 GB1L2 Cross-framework verified
Apple M1OllamaQ2_K72.5 tok/s6.72 GB1L2 Cross-framework verified
Apple M1 ProOllamaIQ3_XS98.6 tok/s7.31 GB1L2 Cross-framework verified
Apple M1 ProStrataFP1658.7 tok/s17.64 GB1L2 Cross-framework verified
Apple M1 Maxllama.cppQ2_K145.2 tok/s6.72 GB1L2 Cross-framework verified
Apple M1 MaxOllamaIQ2_M126.4 tok/s6.47 GB1L2 Cross-framework verified
Apple M1 MaxOllamaQ8_064.7 tok/s11.76 GB1L2 Cross-framework verified
Apple M2 ProvLLMFP1648.5 tok/s17.64 GB1L2 Cross-framework verified
Apple M2 ProvLLMQ6_K73 tok/s10.5 GB1L2 Cross-framework verified
Apple M2 ProStrataFP1665.2 tok/s17.64 GB1L2 Cross-framework verified
Apple M2 UltraOllamaFP1664.1 tok/s17.64 GB1L2 Cross-framework verified
Apple M2 UltraOllamaIQ2_XS161.9 tok/s6.13 GB1L2 Cross-framework verified
Apple M2 UltraStrataQ3_K_L200.6 tok/s8.15 GB1L2 Cross-framework verified
Apple M2 UltraStrataQ5_K_S162.1 tok/s9.66 GB1L2 Cross-framework verified
Qualcomm Snapdragon X ElitevLLMIQ4_XS66.1 tok/s8.32 GB1L2 Cross-framework verified
Qualcomm Snapdragon X ElitevLLMQ3_K_M74.2 tok/s7.81 GB1L2 Cross-framework verified
Qualcomm Snapdragon X EliteStrataFP1639.4 tok/s15.2 GB1L2 Cross-framework verified
Qualcomm Snapdragon 7 Gen 4OllamaQ2_K58.8 tok/s6.72 GB1L2 Cross-framework verified

90 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion