A DeepSeek-V4 MoE model: 284B total parameters with 13B activated, 1M context, MIT licensed. Its hybrid attention (CSA + HCA) cuts long-context KV cache to roughly a tenth of V3.2's.
0
Quantized builds
32
Benchmark records
1
Discussions
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
32 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| NVIDIA RTX 4080 Super | vLLM | Q4_K_M | 3.6 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | vLLM | IQ3_S | 5.4 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | vLLM | IQ4_XS | 4.1 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | Q4_K_M | 3.3 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | IQ3_S | 5 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4080 Super | llama.cpp | IQ4_XS | 3.7 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | vLLM | Q4_K_M | 4.4 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | vLLM | IQ3_S | 6.6 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | vLLM | IQ4_XS | 5 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | Q4_K_M | 4.1 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | IQ3_S | 6.1 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | llama.cpp | IQ4_XS | 4.6 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | Q4_K_M | 8.8 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | IQ4_XS | 9.9 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | Q4_K_M | 8.1 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | IQ3_S | 12.2 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | llama.cpp | IQ4_XS | 9.1 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | vLLM | Q4_K_M | 5 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | vLLM | IQ4_XS | 5.6 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | Q4_K_M | 4.6 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | IQ3_S | 6.8 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | llama.cpp | IQ4_XS | 5.1 tok/s | 24 GB | 1 | L0 Self-reported |
| Apple M5 Max | llama.cpp | IQ3_S | 4.2 tok/s | 115.5 GB | 1 | L0 Self-reported |
| Apple M4 Max | llama.cpp | IQ3_S | 3.7 tok/s | 115.5 GB | 1 | L0 Self-reported |
| NVIDIA RTX 3060 12GB | llama.cpp | IQ2_XXS | 3.5 tok/s | 12 GB | 1 | L0 Self-reported |
| NVIDIA RTX 3090 | llama.cpp | IQ3_S | 6.3 tok/s | 24 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 Ti | Strata | IQ2_XS | 9.3 tok/s | 16 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5070 | llama.cpp | IQ2_XS | 5.5 tok/s | 12 GB | 1 | L0 Self-reported |
| NVIDIA RTX 5090 | vLLM | IQ3_S | 13.2 tok/s | 32 GB | 1 | L0 Self-reported |
| NVIDIA RTX 4090 | vLLM | IQ3_S | 7.5 tok/s | 24 GB | 1 | L0 Self-reported |
| 2x NVIDIA RTX 5090 | llama.cpp | UD-IQ2_XXS | 19.4 tok/s | 51.17 GB | 1 | L0 Self-reported |
| 4x NVIDIA RTX 3090 | llama.cpp | UD-IQ2_XXS | 35.7 tok/s | 87.16 GB | 1 | L0 Self-reported |
32 of these records are not linked to a specific quantized build; the full set is on the search page.