Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX PRO 6000 Blackwell · 96GB · vLLM
Other
Today
9.3 tok/s
Decode
69.8 tok/s
Prefill
1.43 s
TTFT
96 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
2x NVIDIA RTX 5090 · 64GB · vLLM
Other
Today
— tok/s
Decode
— tok/s
Prefill
— s
TTFT
64 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5070 SUPER · 18GB · Strata
Other
Today
7.8 tok/s
Decode
58.5 tok/s
Prefill
1.71 s
TTFT
18 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 SUPER · 24GB · llama.cpp
Other
Today
6.4 tok/s
Decode
48 tok/s
Prefill
2.08 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
Today
11.1 tok/s
Decode
83.3 tok/s
Prefill
1.2 s
TTFT
32 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
Today
6.8 tok/s
Decode
51 tok/s
Prefill
1.96 s
TTFT
24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
6 results