Qwen3.6-35B-A3B

35BChatby Qwen Start a discussion

0

Quantized builds

34

Benchmark records

1

Discussions

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

34 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX Sparkllama.cppQ5_K_M78.8 tok/s70.45 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ8_057.8 tok/s86.62 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ8_052.4 tok/s86.62 GB1L2 Cross-framework verified
NVIDIA RTX 5090OllamaIQ1_S346.5 tok/s30.4 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395vLLMQ3_K_S72.5 tok/s49.88 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395vLLMQ6_K44.3 tok/s75.6 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaQ5_K_M45.6 tok/s70.45 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppIQ3_XS159.3 tok/s47.67 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppQ6_K90.4 tok/s75.6 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ3_K_L131.3 tok/s55.02 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ5_K_S105 tok/s68.25 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppQ5_K_S169.2 tok/s68.25 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ3_K_M206.7 tok/s52.08 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMIQ4_XS288.7 tok/s56.49 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMQ3_K_M355.6 tok/s52.08 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3OllamaIQ2_M321.9 tok/s40.32 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3OllamaQ8_0296.9 tok/s76 GB1L2 Cross-framework verified
NVIDIA H20vLLMQ3_K_S118.5 tok/s49.88 GB1L2 Cross-framework verified
NVIDIA H20vLLMQ8_066.6 tok/s86.62 GB1L2 Cross-framework verified
NVIDIA H20OllamaQ3_K_M90.2 tok/s52.08 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBvLLMIQ3_XS319.3 tok/s47.67 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBOllamaQ5_K_S301.4 tok/s68.25 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppIQ1_S250.5 tok/s32.97 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppQ8_0115 tok/s76 GB1L2 Cross-framework verified
Apple M2 UltraOllamaQ2_K217.1 tok/s42.52 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppIQ3_XS117.7 tok/s47.67 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ4_K_M88.7 tok/s58.9 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaIQ2_XS128.3 tok/s37.38 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ4_K_M85.4 tok/s58.9 GB1L2 Cross-framework verified
Apple M3 Maxllama.cppIQ2_XS89.4 tok/s34.2 GB1L2 Cross-framework verified
Apple M3 MaxvLLMIQ1_S102.6 tok/s32.97 GB1L2 Cross-framework verified
Apple M4 Maxllama.cppIQ3_XS121.6 tok/s47.67 GB1L2 Cross-framework verified
Apple M4 Maxllama.cppQ4_K_S111 tok/s59.43 GB1L2 Cross-framework verified
Apple M4 MaxOllamaQ3_K_L102.3 tok/s55.02 GB1L2 Cross-framework verified

34 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

View all discussions