Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX PRO 5000 Blackwell · 48GB · vLLM
Other
Today
3.7 tok/s
Decode
27.8 tok/s
Prefill
3.6 s
TTFT
48 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
2x NVIDIA RTX 5090 · 64GB · vLLM
Other
Today
— tok/s
Decode
— tok/s
Prefill
— s
TTFT
64 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · llama.cpp
Other
Today
1.5 tok/s
Decode
11.3 tok/s
Prefill
8.85 s
TTFT
128 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 SUPER · 24GB · llama.cpp
Other
Today
3.4 tok/s
Decode
25.5 tok/s
Prefill
3.92 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
4.9 tok/s
Decode
36.8 tok/s
Prefill
2.72 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
Today
3.3 tok/s
Decode
24.8 tok/s
Prefill
4.03 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
6 results