Benchmark search

Try these searches

Found 40 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

ModelDeepSeek-V4.1-Flash

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA RTX PRO 4000 Blackwell · 24GB · SGLang

Other

2026-10-06

2.1 tok/s

Decode

15.8 tok/s

Prefill

6.33 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

NVIDIA RTX PRO 4000 Blackwell · 24GB · SGLang

Other

2026-10-06

1.8 tok/s

Decode

13.5 tok/s

Prefill

7.41 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA RTX PRO 4000 Blackwell · 24GB · vLLM

Other

2026-10-06

2.2 tok/s

Decode

16.5 tok/s

Prefill

6.06 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

NVIDIA RTX PRO 4000 Blackwell · 24GB · vLLM

Other

2026-10-06

1.8 tok/s

Decode

13.5 tok/s

Prefill

7.41 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA RTX PRO 5000 Blackwell · 48GB · SGLang

Other

2026-10-06

4.3 tok/s

Decode

32.3 tok/s

Prefill

3.1 s

TTFT

48 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

NVIDIA RTX PRO 5000 Blackwell · 48GB · SGLang

Other

2026-10-06

3.6 tok/s

Decode

27 tok/s

Prefill

3.7 s

TTFT

48 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA RTX PRO 5000 Blackwell · 48GB · vLLM

Other

2026-10-06

4.4 tok/s

Decode

33 tok/s

Prefill

3.03 s

TTFT

48 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA DGX Spark · 128GB · SGLang

Other

2026-10-06

0.9 tok/s

Decode

6.8 tok/s

Prefill

14.71 s

TTFT

128 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

NVIDIA DGX Spark · 128GB · SGLang

Other

2026-10-06

0.7 tok/s

Decode

5.3 tok/s

Prefill

18.87 s

TTFT

128 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

NVIDIA DGX Spark · 128GB · vLLM

Other

2026-10-06

0.9 tok/s

Decode

6.8 tok/s

Prefill

14.71 s

TTFT

128 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

NVIDIA DGX Spark · 128GB · vLLM

Other

2026-10-06

0.8 tok/s

Decode

6 tok/s

Prefill

16.67 s

TTFT

128 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

2x NVIDIA RTX 5070 Ti · 32GB · SGLang

Other

2026-10-06

— tok/s

Decode

— tok/s

Prefill

— s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

2x NVIDIA RTX 5070 Ti · 32GB · SGLang

Other

2026-10-06

— tok/s

Decode

— tok/s

Prefill

— s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S SGLang 0.4.4 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

2x NVIDIA RTX 5070 Ti · 32GB · vLLM

Other

2026-10-06

— tok/s

Decode

— tok/s

Prefill

— s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

2x NVIDIA RTX 5070 Ti · 32GB · vLLM

Other

2026-10-06

— tok/s

Decode

— tok/s

Prefill

— s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

Apple M5 Ultra · 512GB · llama.cpp

Other

2026-10-06

3.6 tok/s

Decode

27 tok/s

Prefill

3.7 s

TTFT

258.1 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

Apple M5 Ultra · 512GB · llama.cpp

Other

2026-10-06

3 tok/s

Decode

22.5 tok/s

Prefill

4.44 s

TTFT

309.6 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

Apple M5 Pro · 64GB · llama.cpp

Other

2026-10-06

0.9 tok/s

Decode

6.8 tok/s

Prefill

14.71 s

TTFT

64 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ3_S

L0 Self-reported

Apple M5 Pro · 64GB · llama.cpp

Other

2026-10-06

0.8 tok/s

Decode

6 tok/s

Prefill

16.67 s

TTFT

64 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

DeepSeek-V4.1-Flash · IQ2_XS

L0 Self-reported

Apple M4 Pro · 64GB · llama.cpp

Other

2026-10-06

0.8 tok/s

Decode

6 tok/s

Prefill

16.67 s

TTFT

64 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ2_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

40 results

Related models
Related hardware
Related frameworks
Related quantizations