Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 5080 SUPER · 24GB · vLLM
Other
Today
36.6 tok/s
Decode
274.5 tok/s
Prefill
0.36 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 SUPER · 24GB · vLLM
Other
Today
44.7 tok/s
Decode
335.3 tok/s
Prefill
0.3 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 SUPER · 24GB · llama.cpp
Other
Today
33.6 tok/s
Decode
252 tok/s
Prefill
0.4 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 SUPER · 24GB · llama.cpp
Other
Today
41 tok/s
Decode
307.5 tok/s
Prefill
0.33 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · vLLM
Other
Today
29.3 tok/s
Decode
219.8 tok/s
Prefill
0.45 s
TTFT
18 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · vLLM
Other
Today
35.8 tok/s
Decode
268.5 tok/s
Prefill
0.37 s
TTFT
18 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · llama.cpp
Other
Today
26.9 tok/s
Decode
201.8 tok/s
Prefill
0.5 s
TTFT
18 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · llama.cpp
Other
Today
32.8 tok/s
Decode
246 tok/s
Prefill
0.41 s
TTFT
18 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
Today
29.3 tok/s
Decode
219.8 tok/s
Prefill
0.45 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
Today
35.8 tok/s
Decode
268.5 tok/s
Prefill
0.37 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
Today
26.9 tok/s
Decode
201.8 tok/s
Prefill
0.5 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
Today
32.8 tok/s
Decode
246 tok/s
Prefill
0.41 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
58.6 tok/s
Decode
439.5 tok/s
Prefill
0.23 s
TTFT
26.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
71.5 tok/s
Decode
536.3 tok/s
Prefill
0.19 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
53.8 tok/s
Decode
403.5 tok/s
Prefill
0.25 s
TTFT
26.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
Today
65.6 tok/s
Decode
492 tok/s
Prefill
0.2 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
Today
33 tok/s
Decode
247.5 tok/s
Prefill
0.4 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
Today
40.2 tok/s
Decode
301.5 tok/s
Prefill
0.33 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
30.2 tok/s
Decode
226.5 tok/s
Prefill
0.44 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
36.9 tok/s
Decode
276.8 tok/s
Prefill
0.36 s
TTFT
21.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
20 results