Benchmark search

Try these searches

Found 25 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

ModelQwen3.5-35B-A3B

Qwen3.5-35B-A3B · IQ2_M

L2 Cross-framework verified

Apple M3 Max · 36GB · Ollama

Other

2026-10-10

78.91 tok/s

Decode

300.73 tok/s

Prefill

2.1 s

TTFT

34.2 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ2_M Ollama v0.25 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ2_M

L2 Cross-framework verified

Apple M3 Max · 36GB · Strata

Other

2026-10-10

121.51 tok/s

Decode

396.4 tok/s

Prefill

2.07 s

TTFT

34.2 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ2_M Strata v0.34 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ1_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

2026-10-10

227.85 tok/s

Decode

830.74 tok/s

Prefill

1.34 s

TTFT

30.4 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 IQ1_S vLLM v0.31

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ4_XS

L2 Cross-framework verified

Apple M4 Max · 128GB · vLLM

Other

2026-10-10

115.28 tok/s

Decode

541.34 tok/s

Prefill

1.41 s

TTFT

58.04 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ4_XS vLLM v0.28 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q4_K_S

L2 Cross-framework verified

Apple M4 Max · 128GB · vLLM

Other

2026-10-10

115.9 tok/s

Decode

629 tok/s

Prefill

2.35 s

TTFT

61.07 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A Q4_K_S vLLM v0.44

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ2_XS

L2 Cross-framework verified

Apple M4 Max · 128GB · Ollama

Other

2026-10-10

138.01 tok/s

Decode

631.53 tok/s

Prefill

2.31 s

TTFT

38.39 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ2_XS Ollama v0.39 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q6_K

L0 Self-reported

NVIDIA RTX 4080 Super · 16GB · llama.cpp

Other

2026-10-06

18.2 tok/s

Decode

136.5 tok/s

Prefill

0.73 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q6_K llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q5_K_S

L0 Self-reported

NVIDIA RTX 4080 Super · 16GB · llama.cpp

Other

2026-10-06

23.6 tok/s

Decode

177 tok/s

Prefill

0.56 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ3_S

L0 Self-reported

NVIDIA RTX 4080 Super · 16GB · llama.cpp

Other

2026-10-06

39.3 tok/s

Decode

294.8 tok/s

Prefill

0.34 s

TTFT

15.1 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ4_XS

L0 Self-reported

NVIDIA RTX 4080 Super · 16GB · llama.cpp

Other

2026-10-06

29.6 tok/s

Decode

222 tok/s

Prefill

0.45 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ4_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q6_K

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

22.1 tok/s

Decode

165.8 tok/s

Prefill

0.6 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q6_K llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q5_K_S

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

28.8 tok/s

Decode

216 tok/s

Prefill

0.46 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ3_S

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

47.9 tok/s

Decode

359.3 tok/s

Prefill

0.28 s

TTFT

15.1 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ4_XS

L0 Self-reported

NVIDIA RTX 5070 Ti · 16GB · llama.cpp

Other

2026-10-06

36 tok/s

Decode

270 tok/s

Prefill

0.37 s

TTFT

16 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ4_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q6_K

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

44.2 tok/s

Decode

331.5 tok/s

Prefill

0.3 s

TTFT

32 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q6_K llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q5_K_S

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

57.5 tok/s

Decode

431.3 tok/s

Prefill

0.23 s

TTFT

24.8 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ3_S

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

95.7 tok/s

Decode

717.8 tok/s

Prefill

0.14 s

TTFT

15.1 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ3_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · IQ4_XS

L0 Self-reported

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-06

72 tok/s

Decode

540 tok/s

Prefill

0.19 s

TTFT

19.9 GB

VRAM

Config (latest run)
Linux CUDA 12.6 IQ4_XS llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q6_K

L0 Self-reported

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-06

24.9 tok/s

Decode

186.8 tok/s

Prefill

0.54 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q6_K llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

Qwen3.5-35B-A3B · Q5_K_S

L0 Self-reported

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-06

32.4 tok/s

Decode

243 tok/s

Prefill

0.41 s

TTFT

24 GB

VRAM

Config (latest run)
Linux CUDA 12.6 Q5_K_S llama.cpp b4200 Flash Attention

Sources: Other · 1 runs · 1 independent sources

25 results

Related models
Related hardware
Related frameworks
Related quantizations