Qwen3.5-0.8B · Q6_K
L2 Cross-framework verifiedNVIDIA RTX Spark · 128GB · llama.cpp
Other
2026-10-20
94.74 tok/s
Decode
304.72 tok/s
Prefill
2.7 s
TTFT
3.78 GB
VRAM
Config (latest run)
Sources: Other · 1 runs · 1 independent sources
Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX Spark · 128GB · llama.cpp
Other
2026-10-20
94.74 tok/s
Decode
304.72 tok/s
Prefill
2.7 s
TTFT
3.78 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX Spark · 128GB · llama.cpp
Other
2026-10-20
76.23 tok/s
Decode
286.92 tok/s
Prefill
2.87 s
TTFT
4.03 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX Spark · 128GB · Ollama
Other
2026-10-20
57.1 tok/s
Decode
306 tok/s
Prefill
2.75 s
TTFT
5.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX Spark · 128GB · Ollama
Other
2026-10-20
79.06 tok/s
Decode
361.81 tok/s
Prefill
0.86 s
TTFT
4.03 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
2026-10-19
348.73 tok/s
Decode
1,496.87 tok/s
Prefill
0.87 s
TTFT
3.41 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
2026-10-19
196.67 tok/s
Decode
827.33 tok/s
Prefill
1.89 s
TTFT
4.03 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
2026-10-19
312 tok/s
Decode
1,593.74 tok/s
Prefill
0.55 s
TTFT
3.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
2026-10-19
157.88 tok/s
Decode
604.44 tok/s
Prefill
0.39 s
TTFT
3.61 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Ollama
Other
2026-10-19
246.2 tok/s
Decode
1,112.28 tok/s
Prefill
2.3 s
TTFT
2.97 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
2026-10-19
280.58 tok/s
Decode
1,056.43 tok/s
Prefill
1.71 s
TTFT
2.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
2026-10-19
171.2 tok/s
Decode
687.99 tok/s
Prefill
0.81 s
TTFT
3.48 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-19
316.53 tok/s
Decode
1,148.64 tok/s
Prefill
2.15 s
TTFT
3.61 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
2026-10-19
315.13 tok/s
Decode
1,204.27 tok/s
Prefill
1.54 s
TTFT
3.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · vLLM
Other
2026-10-19
323.11 tok/s
Decode
1,397.43 tok/s
Prefill
0.23 s
TTFT
3.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
2026-10-19
368.6 tok/s
Decode
1,525.44 tok/s
Prefill
1.58 s
TTFT
3.41 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
2026-10-19
319.12 tok/s
Decode
1,528.98 tok/s
Prefill
1.41 s
TTFT
3.66 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
2026-10-19
72.92 tok/s
Decode
274.88 tok/s
Prefill
1.59 s
TTFT
5.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
2026-10-19
132.45 tok/s
Decode
600.24 tok/s
Prefill
1.08 s
TTFT
3.61 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · Ollama
Other
2026-10-19
138.92 tok/s
Decode
607.45 tok/s
Prefill
1.83 s
TTFT
3.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-19
302.38 tok/s
Decode
993.94 tok/s
Prefill
0.65 s
TTFT
3.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
235 results