Benchmark search

Try these searches

Found 179 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

ModelQwen3.8-9B

Qwen3.8-9B · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 4090 · 24GB · vLLM

Other

2026-10-19

144.28 tok/s

Decode

508.23 tok/s

Prefill

1.4 s

TTFT

14.38 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q3_K_S vLLM v0.22 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q4_K_M

L2 Cross-framework verified

NVIDIA RTX 4090 · 24GB · Strata

Other

2026-10-19

168.91 tok/s

Decode

879.31 tok/s

Prefill

1.68 s

TTFT

17.6 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.8 Q4_K_M Strata v0.31 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · IQ2_XS

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · llama.cpp

Other

2026-10-19

104.18 tok/s

Decode

429.42 tok/s

Prefill

1.38 s

TTFT

11.17 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.6 IQ2_XS llama.cpp v0.29 Flash Attention

Sources: Other · 2 runs · 1 independent sources

Qwen3.8-9B · Q8_0

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · llama.cpp

Other

2026-10-19

44.9 tok/s

Decode

150.43 tok/s

Prefill

1.34 s

TTFT

22.8 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.4 Q8_0 llama.cpp v0.35

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · IQ3_XS

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · Strata

Other

2026-10-19

134.6 tok/s

Decode

685.06 tok/s

Prefill

1.83 s

TTFT

13.82 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 IQ3_XS Strata v0.22 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_L

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · Strata

Other

2026-10-19

103.74 tok/s

Decode

393.26 tok/s

Prefill

2.38 s

TTFT

15.71 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.4 Q3_K_L Strata v0.38

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_L

L2 Cross-framework verified

AMD Radeon RX 7900 XTX · 24GB · llama.cpp

Other

2026-10-19

79 tok/s

Decode

351.53 tok/s

Prefill

0.12 s

TTFT

15.71 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.8 Q3_K_L llama.cpp v0.35

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_S

L2 Cross-framework verified

AMD Radeon RX 7900 XTX · 24GB · Ollama

Other

2026-10-19

81.43 tok/s

Decode

433.18 tok/s

Prefill

0.54 s

TTFT

14.38 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q3_K_S Ollama v0.50 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q8_0

L2 Cross-framework verified

AMD Radeon RX 7900 XTX · 24GB · Ollama

Other

2026-10-19

45.02 tok/s

Decode

207.48 tok/s

Prefill

0.21 s

TTFT

22.8 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 Q8_0 Ollama v0.33 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · IQ2_XS

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-19

304.42 tok/s

Decode

1,543.97 tok/s

Prefill

0.28 s

TTFT

11.17 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.8 IQ2_XS llama.cpp v0.30

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-19

289.63 tok/s

Decode

1,418.95 tok/s

Prefill

1.71 s

TTFT

14.38 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 Q3_K_S llama.cpp v0.30 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q2_K

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-19

312.49 tok/s

Decode

1,617.22 tok/s

Prefill

2.01 s

TTFT

12.5 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.4 Q2_K vLLM v0.21 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q8_0

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-19

156.33 tok/s

Decode

685.96 tok/s

Prefill

1.11 s

TTFT

23.84 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q8_0 vLLM v0.53 Flash Attention

Sources: Other · 2 runs · 1 independent sources

Qwen3.8-9B · IQ3_S

L2 Cross-framework verified

NVIDIA RTX 5080 · 16GB · llama.cpp

Other

2026-10-19

200.28 tok/s

Decode

716.67 tok/s

Prefill

1.97 s

TTFT

14.57 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 IQ3_S llama.cpp v0.25 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q2_K

L2 Cross-framework verified

NVIDIA RTX 5080 · 16GB · vLLM

Other

2026-10-19

238.4 tok/s

Decode

951.3 tok/s

Prefill

2.01 s

TTFT

12.5 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q2_K vLLM v0.50 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q2_K

L2 Cross-framework verified

NVIDIA RTX 3060 12GB · 12GB · Ollama

Other

2026-10-19

64.94 tok/s

Decode

300.22 tok/s

Prefill

0.34 s

TTFT

11.4 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 Q2_K Ollama v0.38 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · IQ2_M

L2 Cross-framework verified

NVIDIA RTX 4080 Super · 16GB · vLLM

Other

2026-10-19

181.18 tok/s

Decode

718.16 tok/s

Prefill

1.97 s

TTFT

11.93 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 IQ2_M vLLM v0.32 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_M

L2 Cross-framework verified

NVIDIA RTX 4080 Super · 16GB · Strata

Other

2026-10-19

213.25 tok/s

Decode

900.93 tok/s

Prefill

2 s

TTFT

14.95 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q3_K_M Strata v0.41 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 4080 Super · 16GB · Strata

Other

2026-10-19

210.74 tok/s

Decode

935.82 tok/s

Prefill

0.74 s

TTFT

14.38 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.8 Q3_K_S Strata v0.39 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.8-9B · Q4_K_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

2026-10-19

26.52 tok/s

Decode

94.42 tok/s

Prefill

0.62 s

TTFT

17.6 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 Q4_K_M llama.cpp v0.44

Sources: Other · 1 runs · 1 independent sources

179 results

Related models
Related hardware
Related frameworks
Related quantizations