Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
323.97 tok/s
Decode
1,188.09 tok/s
Prefill
2.05 s
TTFT
3.96 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
306.65 tok/s
Decode
1,436.24 tok/s
Prefill
1.31 s
TTFT
4.06 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
Today
287.87 tok/s
Decode
1,066.63 tok/s
Prefill
0.32 s
TTFT
3.6 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
Today
285.75 tok/s
Decode
1,116.76 tok/s
Prefill
0.6 s
TTFT
4.06 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
227.45 tok/s
Decode
862.55 tok/s
Prefill
0.46 s
TTFT
4.67 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
175.42 tok/s
Decode
578.57 tok/s
Prefill
0.86 s
TTFT
5.31 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Strata
Other
Today
215.56 tok/s
Decode
902.66 tok/s
Prefill
0.24 s
TTFT
5.67 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · vLLM
Other
Today
309.15 tok/s
Decode
1,109.54 tok/s
Prefill
2.17 s
TTFT
3.81 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · vLLM
Other
Today
226.47 tok/s
Decode
1,116.37 tok/s
Prefill
0.18 s
TTFT
4.67 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · Ollama
Other
Today
96.85 tok/s
Decode
510.37 tok/s
Prefill
0.44 s
TTFT
8.7 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · Ollama
Other
Today
188.36 tok/s
Decode
704.89 tok/s
Prefill
1.31 s
TTFT
4.75 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M3 Max · 36GB · Strata
Other
Today
239.13 tok/s
Decode
849.25 tok/s
Prefill
1.36 s
TTFT
5.31 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
Today
234.47 tok/s
Decode
1,079.52 tok/s
Prefill
2.39 s
TTFT
4.32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
Today
178.72 tok/s
Decode
842.7 tok/s
Prefill
0.98 s
TTFT
5.03 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
Today
196.24 tok/s
Decode
705.31 tok/s
Prefill
0.21 s
TTFT
4.53 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
Today
111.35 tok/s
Decode
362.65 tok/s
Prefill
0.89 s
TTFT
6.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Strata
Other
Today
249 tok/s
Decode
1,211.72 tok/s
Prefill
2.07 s
TTFT
4.88 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
353.77 tok/s
Decode
1,906.09 tok/s
Prefill
1.04 s
TTFT
5.42 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
345.13 tok/s
Decode
1,779.25 tok/s
Prefill
0.42 s
TTFT
5.67 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
Today
199.27 tok/s
Decode
1,057.11 tok/s
Prefill
2.17 s
TTFT
8.7 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
41 results