Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
14 days ago
180.81 tok/s
Decode
654.72 tok/s
Prefill
0.19 s
TTFT
10.04 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
14 days ago
136.35 tok/s
Decode
504.8 tok/s
Prefill
0.59 s
TTFT
13.82 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · vLLM
Other
14 days ago
179.71 tok/s
Decode
772.31 tok/s
Prefill
1.05 s
TTFT
11.17 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Ollama
Other
14 days ago
69.84 tok/s
Decode
337.36 tok/s
Prefill
0.56 s
TTFT
16.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Strata
Other
14 days ago
163.87 tok/s
Decode
748 tok/s
Prefill
1.44 s
TTFT
10.04 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · vLLM
Other
14 days ago
94.98 tok/s
Decode
305.39 tok/s
Prefill
2.11 s
TTFT
14.95 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
14 days ago
219.01 tok/s
Decode
766.77 tok/s
Prefill
0.6 s
TTFT
16.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
14 days ago
174.79 tok/s
Decode
860.94 tok/s
Prefill
0.23 s
TTFT
19.11 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
14 days ago
138.97 tok/s
Decode
570.96 tok/s
Prefill
0.28 s
TTFT
23.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · llama.cpp
Other
14 days ago
196.47 tok/s
Decode
707.56 tok/s
Prefill
2.07 s
TTFT
13.82 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · vLLM
Other
14 days ago
70.07 tok/s
Decode
259.24 tok/s
Prefill
0.62 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · Ollama
Other
14 days ago
145.8 tok/s
Decode
516.93 tok/s
Prefill
1.05 s
TTFT
15.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · Strata
Other
14 days ago
244.72 tok/s
Decode
1,076.88 tok/s
Prefill
1.66 s
TTFT
13.82 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · llama.cpp
Other
14 days ago
32.72 tok/s
Decode
137.17 tok/s
Prefill
2.45 s
TTFT
15.71 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
14 days ago
22.65 tok/s
Decode
118.27 tok/s
Prefill
1.87 s
TTFT
21 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · vLLM
Other
14 days ago
19.62 tok/s
Decode
103.78 tok/s
Prefill
2.34 s
TTFT
23.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · llama.cpp
Other
14 days ago
59.51 tok/s
Decode
259.7 tok/s
Prefill
1.23 s
TTFT
19.11 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
14 days ago
30.75 tok/s
Decode
124.45 tok/s
Prefill
0.39 s
TTFT
37.06 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
14 days ago
53.6 tok/s
Decode
253.24 tok/s
Prefill
1.28 s
TTFT
19.11 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · llama.cpp
Other
14 days ago
117.38 tok/s
Decode
386.02 tok/s
Prefill
2.43 s
TTFT
14.38 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
37 results