Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
17 days ago
122.06 tok/s
Decode
409.59 tok/s
Prefill
2.2 s
TTFT
22.09 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
17 days ago
79.26 tok/s
Decode
346.57 tok/s
Prefill
2.5 s
TTFT
22.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
17 days ago
75.68 tok/s
Decode
415.45 tok/s
Prefill
1.01 s
TTFT
22.09 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · vLLM
Other
17 days ago
80.11 tok/s
Decode
318.86 tok/s
Prefill
0.68 s
TTFT
22.09 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · llama.cpp
Other
17 days ago
65.83 tok/s
Decode
346.39 tok/s
Prefill
2.3 s
TTFT
22.8 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
17 days ago
90.76 tok/s
Decode
372.69 tok/s
Prefill
1.03 s
TTFT
16.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
17 days ago
145.18 tok/s
Decode
478.54 tok/s
Prefill
1.69 s
TTFT
30.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · vLLM
Other
17 days ago
227.28 tok/s
Decode
1,043.49 tok/s
Prefill
0.18 s
TTFT
15.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · llama.cpp
Other
17 days ago
208.08 tok/s
Decode
731.51 tok/s
Prefill
1.09 s
TTFT
14.45 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
17 days ago
34.77 tok/s
Decode
161.9 tok/s
Prefill
0.75 s
TTFT
21.21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
17 days ago
29.55 tok/s
Decode
96.92 tok/s
Prefill
2.13 s
TTFT
25.03 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · Ollama
Other
17 days ago
28.23 tok/s
Decode
99.22 tok/s
Prefill
1.42 s
TTFT
25.03 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · llama.cpp
Other
17 days ago
66.61 tok/s
Decode
292.99 tok/s
Prefill
1.37 s
TTFT
22.09 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
17 days ago
63.06 tok/s
Decode
323.43 tok/s
Prefill
0.81 s
TTFT
23.86 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
17 days ago
36.84 tok/s
Decode
127.42 tok/s
Prefill
0.3 s
TTFT
35.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · llama.cpp
Other
17 days ago
77.61 tok/s
Decode
409.9 tok/s
Prefill
2 s
TTFT
29.44 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · Ollama
Other
17 days ago
118.32 tok/s
Decode
453.53 tok/s
Prefill
0.83 s
TTFT
18.27 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · Ollama
Other
17 days ago
65.66 tok/s
Decode
313.69 tok/s
Prefill
0.3 s
TTFT
29.44 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H100 80GB HBM3 · 80GB · vLLM
Other
17 days ago
398.24 tok/s
Decode
1,414.76 tok/s
Prefill
2.13 s
TTFT
23.86 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H100 80GB HBM3 · 80GB · vLLM
Other
17 days ago
364.79 tok/s
Decode
1,787.53 tok/s
Prefill
2.34 s
TTFT
28.56 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
82 results