Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · vLLM
Other
Today
362.35 tok/s
Decode
1,574.76 tok/s
Prefill
0.64 s
TTFT
6.26 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
Today
284.1 tok/s
Decode
1,178.93 tok/s
Prefill
0.55 s
TTFT
4.75 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
Today
281.21 tok/s
Decode
1,389.98 tok/s
Prefill
0.16 s
TTFT
5.12 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
268 tok/s
Decode
1,209.31 tok/s
Prefill
2.46 s
TTFT
5.57 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Strata
Other
Today
145.3 tok/s
Decode
693.51 tok/s
Prefill
0.85 s
TTFT
13.76 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4060 Ti 16GB · 16GB · llama.cpp
Other
Today
326.89 tok/s
Decode
1,104.91 tok/s
Prefill
0.63 s
TTFT
6.26 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4060 Ti 16GB · 16GB · vLLM
Other
Today
350.23 tok/s
Decode
1,317.38 tok/s
Prefill
0.97 s
TTFT
5.57 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · vLLM
Other
Today
253.22 tok/s
Decode
1,074.86 tok/s
Prefill
1.24 s
TTFT
6.26 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Strata
Other
Today
389.24 tok/s
Decode
2,015.9 tok/s
Prefill
1.4 s
TTFT
6.26 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Strata
Other
Today
341.32 tok/s
Decode
1,578.68 tok/s
Prefill
0.54 s
TTFT
7.01 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
325.57 tok/s
Decode
1,657.68 tok/s
Prefill
0.19 s
TTFT
5.57 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
376.12 tok/s
Decode
1,436.24 tok/s
Prefill
2.42 s
TTFT
7.01 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Strata
Other
Today
367.38 tok/s
Decode
1,255.45 tok/s
Prefill
1.35 s
TTFT
7.27 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Strata
Other
Today
289.95 tok/s
Decode
1,022.85 tok/s
Prefill
0.33 s
TTFT
7.77 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · llama.cpp
Other
Today
351.28 tok/s
Decode
1,853.09 tok/s
Prefill
1.2 s
TTFT
4.75 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
Today
362.6 tok/s
Decode
1,325.05 tok/s
Prefill
2.25 s
TTFT
6.76 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
Today
193.44 tok/s
Decode
733.2 tok/s
Prefill
0.65 s
TTFT
6.26 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · llama.cpp
Other
Today
126.84 tok/s
Decode
630.8 tok/s
Prefill
0.66 s
TTFT
7.77 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · vLLM
Other
Today
143.38 tok/s
Decode
521.02 tok/s
Prefill
1.74 s
TTFT
7.96 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 9070 XT · 16GB · llama.cpp
Other
Today
319.75 tok/s
Decode
1,663.47 tok/s
Prefill
1.88 s
TTFT
6.01 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
59 results