Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4080 Super · 16GB · SGLang
Other
2026-10-06
3.1 tok/s
Decode
23.3 tok/s
Prefill
4.29 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · SGLang
Other
2026-10-06
4.7 tok/s
Decode
35.3 tok/s
Prefill
2.83 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · SGLang
Other
2026-10-06
3.5 tok/s
Decode
26.3 tok/s
Prefill
3.8 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
3.2 tok/s
Decode
24 tok/s
Prefill
4.17 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
4.8 tok/s
Decode
36 tok/s
Prefill
2.78 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · vLLM
Other
2026-10-06
3.6 tok/s
Decode
27 tok/s
Prefill
3.7 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · SGLang
Other
2026-10-06
3.8 tok/s
Decode
28.5 tok/s
Prefill
3.51 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · SGLang
Other
2026-10-06
5.7 tok/s
Decode
42.8 tok/s
Prefill
2.34 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · SGLang
Other
2026-10-06
4.3 tok/s
Decode
32.3 tok/s
Prefill
3.1 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
3.9 tok/s
Decode
29.3 tok/s
Prefill
3.41 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
5.9 tok/s
Decode
44.3 tok/s
Prefill
2.26 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 Ti · 16GB · vLLM
Other
2026-10-06
4.4 tok/s
Decode
33 tok/s
Prefill
3.03 s
TTFT
16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · SGLang
Other
2026-10-06
7.6 tok/s
Decode
57 tok/s
Prefill
1.75 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · SGLang
Other
2026-10-06
8.5 tok/s
Decode
63.8 tok/s
Prefill
1.57 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-06
7.8 tok/s
Decode
58.5 tok/s
Prefill
1.71 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-06
11.8 tok/s
Decode
88.5 tok/s
Prefill
1.13 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-06
8.8 tok/s
Decode
66 tok/s
Prefill
1.52 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · SGLang
Other
2026-10-06
4.3 tok/s
Decode
32.3 tok/s
Prefill
3.1 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · SGLang
Other
2026-10-06
6.4 tok/s
Decode
48 tok/s
Prefill
2.08 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · SGLang
Other
2026-10-06
4.8 tok/s
Decode
36 tok/s
Prefill
2.78 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
29 results