Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 5060 SUPER · 12GB · vLLM
Other
Today
29.3 tok/s
Decode
219.8 tok/s
Prefill
0.45 s
TTFT
12 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5060 SUPER · 12GB · vLLM
Other
Today
42.8 tok/s
Decode
321 tok/s
Prefill
0.31 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5060 SUPER · 12GB · vLLM
Other
Today
52.2 tok/s
Decode
391.5 tok/s
Prefill
0.26 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5060 SUPER · 12GB · llama.cpp
Other
Today
26.9 tok/s
Decode
201.8 tok/s
Prefill
0.5 s
TTFT
12 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5060 SUPER · 12GB · llama.cpp
Other
Today
39.3 tok/s
Decode
294.8 tok/s
Prefill
0.34 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5060 SUPER · 12GB · llama.cpp
Other
Today
47.9 tok/s
Decode
359.3 tok/s
Prefill
0.28 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · vLLM
Other
Today
58.6 tok/s
Decode
439.5 tok/s
Prefill
0.23 s
TTFT
13.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · vLLM
Other
Today
85.6 tok/s
Decode
642 tok/s
Prefill
0.16 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · vLLM
Other
Today
104.3 tok/s
Decode
782.3 tok/s
Prefill
0.13 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · llama.cpp
Other
Today
53.8 tok/s
Decode
403.5 tok/s
Prefill
0.25 s
TTFT
13.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · llama.cpp
Other
Today
78.5 tok/s
Decode
588.8 tok/s
Prefill
0.17 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · llama.cpp
Other
Today
95.7 tok/s
Decode
717.8 tok/s
Prefill
0.14 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · vLLM
Other
Today
61.2 tok/s
Decode
459 tok/s
Prefill
0.22 s
TTFT
13.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · vLLM
Other
Today
89.4 tok/s
Decode
670.5 tok/s
Prefill
0.15 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · vLLM
Other
Today
109 tok/s
Decode
817.5 tok/s
Prefill
0.12 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
56.2 tok/s
Decode
421.5 tok/s
Prefill
0.24 s
TTFT
13.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
82 tok/s
Decode
615 tok/s
Prefill
0.16 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
Today
100 tok/s
Decode
750 tok/s
Prefill
0.13 s
TTFT
7.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
Today
58.6 tok/s
Decode
439.5 tok/s
Prefill
0.23 s
TTFT
13.5 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
Today
85.6 tok/s
Decode
642 tok/s
Prefill
0.16 s
TTFT
9.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
42 results