Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
17 days ago
118.71 tok/s
Decode
394.1 tok/s
Prefill
0.47 s
TTFT
17.22 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
17 days ago
177.41 tok/s
Decode
791.84 tok/s
Prefill
1.22 s
TTFT
13.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
17 days ago
157.4 tok/s
Decode
646.64 tok/s
Prefill
1.85 s
TTFT
15.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Ollama
Other
17 days ago
67.97 tok/s
Decode
263.38 tok/s
Prefill
0.42 s
TTFT
17.22 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Strata
Other
17 days ago
155.86 tok/s
Decode
503.14 tok/s
Prefill
1.78 s
TTFT
13.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Strata
Other
17 days ago
101.92 tok/s
Decode
340.28 tok/s
Prefill
1.15 s
TTFT
17.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · vLLM
Other
17 days ago
95.65 tok/s
Decode
325.43 tok/s
Prefill
1.27 s
TTFT
15.88 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
17 days ago
74.72 tok/s
Decode
318.2 tok/s
Prefill
0.55 s
TTFT
17.22 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
17 days ago
353.35 tok/s
Decode
1,757.41 tok/s
Prefill
0.81 s
TTFT
12.52 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
17 days ago
288.35 tok/s
Decode
1,282.34 tok/s
Prefill
1.36 s
TTFT
15.88 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
17 days ago
226.28 tok/s
Decode
869.25 tok/s
Prefill
0.43 s
TTFT
17.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
17 days ago
199.24 tok/s
Decode
946.23 tok/s
Prefill
0.15 s
TTFT
13.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Ollama
Other
17 days ago
204.49 tok/s
Decode
898.01 tok/s
Prefill
0.55 s
TTFT
13.02 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Strata
Other
17 days ago
349.68 tok/s
Decode
1,703.4 tok/s
Prefill
1.59 s
TTFT
9.16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Strata
Other
17 days ago
297.89 tok/s
Decode
1,445.32 tok/s
Prefill
2.26 s
TTFT
14.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · Strata
Other
17 days ago
95.19 tok/s
Decode
317.89 tok/s
Prefill
0.3 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · Ollama
Other
17 days ago
190.92 tok/s
Decode
657.75 tok/s
Prefill
1.17 s
TTFT
12.52 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · Strata
Other
17 days ago
334.52 tok/s
Decode
1,091.81 tok/s
Prefill
2.26 s
TTFT
10.16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
17 days ago
48.9 tok/s
Decode
230.21 tok/s
Prefill
1.68 s
TTFT
10.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
17 days ago
31.8 tok/s
Decode
157.2 tok/s
Prefill
2.46 s
TTFT
17.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
86 results