Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
123.9 tok/s
Decode
634.46 tok/s
Prefill
0.58 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Ollama
Other
Today
113.94 tok/s
Decode
395.79 tok/s
Prefill
0.2 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Ollama
Other
Today
70.56 tok/s
Decode
253.27 tok/s
Prefill
0.65 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
61.22 tok/s
Decode
289.69 tok/s
Prefill
1.04 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Ollama
Other
Today
54.04 tok/s
Decode
184.49 tok/s
Prefill
0.99 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4060 Ti 16GB · 16GB · vLLM
Other
Today
82.77 tok/s
Decode
438.83 tok/s
Prefill
2.43 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4060 Ti 16GB · 16GB · Ollama
Other
Today
90.69 tok/s
Decode
447.55 tok/s
Prefill
1.06 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4060 Ti 16GB · 16GB · Ollama
Other
Today
88.03 tok/s
Decode
388.48 tok/s
Prefill
0.2 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
Today
109.82 tok/s
Decode
547.49 tok/s
Prefill
2.4 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
Today
75 tok/s
Decode
318.12 tok/s
Prefill
1.01 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
Today
56.93 tok/s
Decode
285.15 tok/s
Prefill
2.04 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
253.52 tok/s
Decode
1,057.1 tok/s
Prefill
0.13 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
223.8 tok/s
Decode
1,049.81 tok/s
Prefill
2.15 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Strata
Other
Today
330.44 tok/s
Decode
1,792.04 tok/s
Prefill
2.4 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
Today
124.24 tok/s
Decode
658.04 tok/s
Prefill
1.45 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
Today
85.12 tok/s
Decode
372.82 tok/s
Prefill
0.69 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Strata
Other
Today
128.67 tok/s
Decode
423.79 tok/s
Prefill
1.28 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
Today
55.93 tok/s
Decode
219.18 tok/s
Prefill
2.35 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
Today
58.85 tok/s
Decode
193.51 tok/s
Prefill
1.17 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · vLLM
Other
Today
53.14 tok/s
Decode
228.98 tok/s
Prefill
2.17 s
TTFT
1.05 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
56 results