Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · Ollama
Other
12 days ago
158.5 tok/s
Decode
719.84 tok/s
Prefill
0.55 s
TTFT
13.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Strata
Other
12 days ago
243.76 tok/s
Decode
1,261.12 tok/s
Prefill
2.02 s
TTFT
13.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
12 days ago
121.92 tok/s
Decode
447.63 tok/s
Prefill
1.67 s
TTFT
13.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · vLLM
Other
12 days ago
102.62 tok/s
Decode
342.57 tok/s
Prefill
1.22 s
TTFT
14.53 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · vLLM
Other
12 days ago
133.18 tok/s
Decode
447.04 tok/s
Prefill
1.69 s
TTFT
12.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
12 days ago
71.28 tok/s
Decode
362.47 tok/s
Prefill
1.15 s
TTFT
17.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · vLLM
Other
12 days ago
301.33 tok/s
Decode
1,596.94 tok/s
Prefill
1.8 s
TTFT
10.16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5080 · 16GB · Strata
Other
12 days ago
314.5 tok/s
Decode
1,280.69 tok/s
Prefill
1.1 s
TTFT
14.53 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · vLLM
Other
12 days ago
89.05 tok/s
Decode
461.73 tok/s
Prefill
0.57 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · Ollama
Other
12 days ago
79.42 tok/s
Decode
278.2 tok/s
Prefill
0.3 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4080 Super · 16GB · Ollama
Other
12 days ago
222.5 tok/s
Decode
1,136.51 tok/s
Prefill
1.34 s
TTFT
11.34 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · llama.cpp
Other
12 days ago
153.37 tok/s
Decode
580.58 tok/s
Prefill
1.38 s
TTFT
13.19 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · Ollama
Other
12 days ago
156.39 tok/s
Decode
558.43 tok/s
Prefill
1.17 s
TTFT
10.84 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H100 80GB HBM3 · 80GB · llama.cpp
Other
12 days ago
297.43 tok/s
Decode
1,240.57 tok/s
Prefill
1.94 s
TTFT
12.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H100 80GB HBM3 · 80GB · vLLM
Other
12 days ago
343.31 tok/s
Decode
1,851.05 tok/s
Prefill
0.94 s
TTFT
13.02 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H20 · 97GB · llama.cpp
Other
12 days ago
95.17 tok/s
Decode
436.06 tok/s
Prefill
1.22 s
TTFT
10.16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA H20 · 97GB · vLLM
Other
12 days ago
78.41 tok/s
Decode
259.38 tok/s
Prefill
0.43 s
TTFT
13.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX PRO 6000 Blackwell 96GB · 96GB · vLLM
Other
12 days ago
344.94 tok/s
Decode
1,590.34 tok/s
Prefill
2.16 s
TTFT
12.52 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX PRO 6000 Blackwell 96GB · 96GB · Ollama
Other
12 days ago
387.68 tok/s
Decode
2,077.1 tok/s
Prefill
1.62 s
TTFT
10.16 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Intel Arc B580 · 12GB · llama.cpp
Other
12 days ago
70.76 tok/s
Decode
300.15 tok/s
Prefill
1.11 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
3301 results