Qwen3-8B

8BChatby Qwen Start a discussion

0

Quantized builds

210

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

50 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX 4090llama.cppQ4_K_M142.7 tok/s10.64 GB2L2 Cross-framework verified
NVIDIA RTX 4090vLLMIQ3_S161.1 tok/s13.19 GB1L2 Cross-framework verified
NVIDIA RTX 3090vLLMIQ2_XS113.4 tok/s10.16 GB1L2 Cross-framework verified
NVIDIA RTX 3090OllamaQ2_K98.9 tok/s11.34 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXllama.cppQ5_K_S73.4 tok/s17.22 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXvLLMIQ2_M127.9 tok/s10.84 GB2L2 Cross-framework verified
AMD Radeon RX 7900 XTXvLLMQ2_K108.2 tok/s11.34 GB1L2 Cross-framework verified
NVIDIA RTX 5090OllamaIQ3_S275.4 tok/s13.19 GB1L2 Cross-framework verified
NVIDIA RTX 5090StrataIQ4_XS340.7 tok/s14.53 GB1L2 Cross-framework verified
NVIDIA RTX 5080llama.cppQ3_K_S206.6 tok/s13.02 GB1L2 Cross-framework verified
NVIDIA RTX 5080vLLMIQ2_M240.9 tok/s10.84 GB1L2 Cross-framework verified
NVIDIA RTX 3060 12GBvLLMQ3_K_S73.3 tok/s11.4 GB1L2 Cross-framework verified
NVIDIA RTX 3060 12GBOllamaIQ3_S58.4 tok/s11.4 GB1L2 Cross-framework verified
NVIDIA RTX 4080 Superllama.cppQ3_K_S168.4 tok/s13.02 GB1L2 Cross-framework verified
NVIDIA RTX 4080 SupervLLMIQ2_M214.4 tok/s10.84 GB2L2 Cross-framework verified
AMD Ryzen AI Max+ 395vLLMQ3_K_S38.8 tok/s13.02 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaFP1614.1 tok/s33.18 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaQ3_K_M34.2 tok/s13.52 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppIQ1_S95.3 tok/s9.16 GB1L2 Cross-framework verified
Apple M5 MaxOllamaIQ2_M82.4 tok/s10.84 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ6_K50.4 tok/s18.9 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppQ4_K_M102.1 tok/s15.88 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppQ6_K73.3 tok/s18.9 GB1L2 Cross-framework verified
Apple M5 UltraOllamaIQ3_XS108 tok/s12.52 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ4_K_S102.8 tok/s15.2 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMIQ1_S335.5 tok/s9.16 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3OllamaQ6_K301.7 tok/s18.9 GB1L2 Cross-framework verified
NVIDIA H20llama.cppIQ3_S57.4 tok/s13.19 GB1L2 Cross-framework verified
NVIDIA H20llama.cppQ5_K_S42.5 tok/s17.22 GB1L2 Cross-framework verified
NVIDIA H20StrataIQ3_S87.3 tok/s13.19 GB1L2 Cross-framework verified
NVIDIA H20StrataQ5_K_S59.7 tok/s17.22 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBllama.cppQ2_K305.2 tok/s11.34 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBStrataIQ3_S374.2 tok/s13.19 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBStrataQ3_K_L317.2 tok/s14.2 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppQ3_K_S119.1 tok/s13.02 GB1L2 Cross-framework verified
Apple M2 UltraOllamaQ2_K112.3 tok/s11.34 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ4_K_M51 tok/s15.88 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ5_K_M46.5 tok/s17.72 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ3_K_L51.9 tok/s14.2 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ4_K_M50.4 tok/s15.88 GB1L2 Cross-framework verified
NVIDIA RTX 4090llama.cppIQ1_S187.9 tok/s9.16 GB1L2 Cross-framework verified
NVIDIA RTX 4090llama.cppQ8_087.2 tok/s17.31 GB3L2 Cross-framework verified
NVIDIA RTX 4090vLLMQ2_K164.9 tok/s11.34 GB1L2 Cross-framework verified
NVIDIA RTX 3090vLLMIQ3_XS108.6 tok/s12.52 GB1L2 Cross-framework verified
NVIDIA RTX 3090OllamaIQ4_XS96.7 tok/s14.53 GB2L2 Cross-framework verified
NVIDIA RTX 3090OllamaQ8_048 tok/s21.42 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBllama.cppIQ3_XS120.5 tok/s12.52 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBllama.cppQ2_K129.6 tok/s11.34 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBvLLMQ3_K_L108.1 tok/s14.2 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXOllamaIQ3_XS90.6 tok/s12.52 GB1L2 Cross-framework verified

210 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion