Qwen3.5-35B-A3B · IQ2_M
L2 Cross-framework verifiedApple M3 Max · 36GB · Ollama
Other
2026-10-10
78.91 tok/s
Decode
300.73 tok/s
Prefill
2.1 s
TTFT
34.2 GB
VRAM
Config (latest run)
Sources: Other · 1 runs · 1 independent sources
Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
Apple M3 Max · 36GB · Ollama
Other
2026-10-10
78.91 tok/s
Decode
300.73 tok/s
Prefill
2.1 s
TTFT
34.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · Strata
Other
2026-10-10
121.51 tok/s
Decode
396.4 tok/s
Prefill
2.07 s
TTFT
34.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-10
227.85 tok/s
Decode
830.74 tok/s
Prefill
1.34 s
TTFT
30.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M4 Max · 128GB · vLLM
Other
2026-10-10
115.28 tok/s
Decode
541.34 tok/s
Prefill
1.41 s
TTFT
58.04 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M4 Max · 128GB · vLLM
Other
2026-10-10
115.9 tok/s
Decode
629 tok/s
Prefill
2.35 s
TTFT
61.07 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M4 Max · 128GB · Ollama
Other
2026-10-10
138.01 tok/s
Decode
631.53 tok/s
Prefill
2.31 s
TTFT
38.39 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
18.2 tok/s
Decode
136.5 tok/s
Prefill
0.73 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
23.6 tok/s
Decode
177 tok/s
Prefill
0.56 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
39.3 tok/s
Decode
294.8 tok/s
Prefill
0.34 s
TTFT
15.1 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
29.6 tok/s
Decode
222 tok/s
Prefill
0.45 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
22.1 tok/s
Decode
165.8 tok/s
Prefill
0.6 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
28.8 tok/s
Decode
216 tok/s
Prefill
0.46 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
47.9 tok/s
Decode
359.3 tok/s
Prefill
0.28 s
TTFT
15.1 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
36 tok/s
Decode
270 tok/s
Prefill
0.37 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
44.2 tok/s
Decode
331.5 tok/s
Prefill
0.3 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
57.5 tok/s
Decode
431.3 tok/s
Prefill
0.23 s
TTFT
24.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
95.7 tok/s
Decode
717.8 tok/s
Prefill
0.14 s
TTFT
15.1 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
72 tok/s
Decode
540 tok/s
Prefill
0.19 s
TTFT
19.9 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
2026-10-06
24.9 tok/s
Decode
186.8 tok/s
Prefill
0.54 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
2026-10-06
32.4 tok/s
Decode
243 tok/s
Prefill
0.41 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
25 results