Benchmark search

Try these searches

Found 18 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

ModelQwen3-30B-A3B

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · vLLM

Other

2026-10-06

23.5 tok/s

Decode

176.3 tok/s

Prefill

0.57 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · vLLM

Other

2026-10-06

34.2 tok/s

Decode

256.5 tok/s

Prefill

0.39 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · vLLM

Other

2026-10-06

41.8 tok/s

Decode

313.5 tok/s

Prefill

0.32 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

21.6 tok/s

Decode

162 tok/s

Prefill

0.62 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

31.3 tok/s

Decode

234.8 tok/s

Prefill

0.43 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

38.4 tok/s

Decode

288 tok/s

Prefill

0.35 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-06

47 tok/s

Decode

352.5 tok/s

Prefill

0.28 s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-06

68.3 tok/s

Decode

512.3 tok/s

Prefill

0.2 s

TTFT

22.8 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-06

83.7 tok/s

Decode

627.8 tok/s

Prefill

0.16 s

TTFT

18.7 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

43.1 tok/s

Decode

323.3 tok/s

Prefill

0.31 s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

62.7 tok/s

Decode

470.3 tok/s

Prefill

0.21 s

TTFT

22.8 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

76.8 tok/s

Decode

576 tok/s

Prefill

0.17 s

TTFT

18.7 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 4090 · 24GB · vLLM

Other

2026-10-06

26.4 tok/s

Decode

198 tok/s

Prefill

0.51 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 4090 · 24GB · vLLM

Other

2026-10-06

38.4 tok/s

Decode

288 tok/s

Prefill

0.35 s

TTFT

22.8 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 4090 · 24GB · vLLM

Other

2026-10-06

47.1 tok/s

Decode

353.3 tok/s

Prefill

0.28 s

TTFT

18.7 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M vLLM 0.7.3 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q8_0

L0 Self-reported

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-06

24.3 tok/s

Decode

182.3 tok/s

Prefill

0.55 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q8_0 llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q5_K_M

L0 Self-reported

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-06

35.3 tok/s

Decode

264.8 tok/s

Prefill

0.38 s

TTFT

22.8 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3-30B-A3B · Q4_K_M

L0 Self-reported

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-06

43.2 tok/s

Decode

324 tok/s

Prefill

0.31 s

TTFT

18.7 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q4_K_M llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

18 results

Related models
Related hardware
Related frameworks
Related quantizations