Open-weight vision-language model from Qwen (Alibaba), released 2026-02-24, 36.0B total with ~3B active, Apache-2.0. MoE with 36B total and about 3B active parameters, for setups that are VRAM-bound but want a larger knowledge pool.
0
Quantized builds
25
Benchmark records
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
25 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| Apple M3 Max | Ollama | IQ2_M | 78.9 tok/s | 34.2 GB | 1 | L2 Cross-framework verified |
| Apple M3 Max | Strata | IQ2_M | 121.5 tok/s | 34.2 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 5090 | vLLM | IQ1_S | 227.8 tok/s | 30.4 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | vLLM | IQ4_XS | 115.3 tok/s | 58.04 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | vLLM | Q4_K_S | 115.9 tok/s | 61.07 GB | 1 | L2 Cross-framework verified |
| Apple M4 Max | Ollama | IQ2_XS | 138 tok/s | 38.39 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX 4080 Super | llama.cpp | Q6_K | 18.2 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | Q5_K_S | 23.6 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | IQ3_S | 39.3 tok/s | 15.1 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | IQ4_XS | 29.6 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | Q6_K | 22.1 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | Q5_K_S | 28.8 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | IQ3_S | 47.9 tok/s | 15.1 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | IQ4_XS | 36 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | Q6_K | 44.2 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | Q5_K_S | 57.5 tok/s | 24.8 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | IQ3_S | 95.7 tok/s | 15.1 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | IQ4_XS | 72 tok/s | 19.9 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q6_K | 24.9 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q5_K_S | 32.4 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | IQ3_S | 53.9 tok/s | 15.1 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | IQ4_XS | 40.5 tok/s | 19.9 GB | 1 | L0 Self-reported |
| Apple M5 Max | llama.cpp | Q4_K_M | 21.9 tok/s | 22.4 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | Q4_K_M | 69.6 tok/s | 22.4 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q4_K_M | 35.9 tok/s | 22.4 GB | 1 | L0 Self-reported |
25 of these records are not linked to a specific quantized build; the full set is on the search page.
Community discussion
No discussions yet — start the first one.
Start a discussion