Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
3.6 tok/s
Decode
27 tok/s
Prefill
3.7 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
5.4 tok/s
Decode
40.5 tok/s
Prefill
2.47 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
4.1 tok/s
Decode
30.7 tok/s
Prefill
3.26 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
3.3 tok/s
Decode
24.8 tok/s
Prefill
4.03 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
5 tok/s
Decode
37.5 tok/s
Prefill
2.67 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
2026-10-06
3.7 tok/s
Decode
27.8 tok/s
Prefill
3.6 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
4.4 tok/s
Decode
33 tok/s
Prefill
3.03 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
6.6 tok/s
Decode
49.5 tok/s
Prefill
2.02 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
5 tok/s
Decode
37.5 tok/s
Prefill
2.67 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
4.1 tok/s
Decode
30.7 tok/s
Prefill
3.26 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
6.1 tok/s
Decode
45.8 tok/s
Prefill
2.18 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · llama.cpp
Other
2026-10-06
4.6 tok/s
Decode
34.5 tok/s
Prefill
2.9 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-06
8.8 tok/s
Decode
66 tok/s
Prefill
1.52 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-06
9.9 tok/s
Decode
74.3 tok/s
Prefill
1.35 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
8.1 tok/s
Decode
60.8 tok/s
Prefill
1.64 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
12.2 tok/s
Decode
91.5 tok/s
Prefill
1.09 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-06
9.1 tok/s
Decode
68.3 tok/s
Prefill
1.46 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
2026-10-06
5 tok/s
Decode
37.5 tok/s
Prefill
2.67 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
2026-10-06
5.6 tok/s
Decode
42 tok/s
Prefill
2.38 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
2026-10-06
4.6 tok/s
Decode
34.5 tok/s
Prefill
2.9 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
32 results