gemma-4-12B-it · Q3_K_L
L2 Cross-framework verifiedNVIDIA RTX 4090 · 24GB · llama.cpp
Other
2026-10-19
114.65 tok/s
Decode
473.3 tok/s
Prefill
2.16 s
TTFT
20.24 GB
VRAM
Config (latest run)
Sources: Other · 1 runs · 1 independent sources
Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 4090 · 24GB · llama.cpp
Other
2026-10-19
114.65 tok/s
Decode
473.3 tok/s
Prefill
2.16 s
TTFT
20.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Ollama
Other
2026-10-19
87.43 tok/s
Decode
399.69 tok/s
Prefill
1.92 s
TTFT
22.76 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 4090 · 24GB · Ollama
Other
2026-10-19
70.15 tok/s
Decode
334.66 tok/s
Prefill
1.31 s
TTFT
22.8 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · llama.cpp
Other
2026-10-19
79.26 tok/s
Decode
473.41 tok/s
Prefill
1.21 s
TTFT
15.28 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 3090 · 24GB · Ollama
Other
2026-10-19
64.09 tok/s
Decode
302.97 tok/s
Prefill
1.12 s
TTFT
20.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · vLLM
Other
2026-10-19
77.87 tok/s
Decode
405.9 tok/s
Prefill
0.18 s
TTFT
19.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Radeon RX 7900 XTX · 24GB · Ollama
Other
2026-10-19
92.04 tok/s
Decode
422.92 tok/s
Prefill
2.03 s
TTFT
12.68 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-19
262.05 tok/s
Decode
997.54 tok/s
Prefill
1.68 s
TTFT
14.2 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-19
125.52 tok/s
Decode
751.15 tok/s
Prefill
0.88 s
TTFT
21.95 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
2026-10-19
236.13 tok/s
Decode
819.72 tok/s
Prefill
1.31 s
TTFT
18.48 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · Ollama
Other
2026-10-19
165.12 tok/s
Decode
690.64 tok/s
Prefill
1.85 s
TTFT
24.78 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 3060 12GB · 12GB · Strata
Other
2026-10-19
99.13 tok/s
Decode
497.2 tok/s
Prefill
0.34 s
TTFT
11.4 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · llama.cpp
Other
2026-10-19
32.57 tok/s
Decode
131.77 tok/s
Prefill
2.15 s
TTFT
15.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · llama.cpp
Other
2026-10-19
22.71 tok/s
Decode
109.92 tok/s
Prefill
0.47 s
TTFT
25.54 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · Ollama
Other
2026-10-19
26.82 tok/s
Decode
137.08 tok/s
Prefill
0.81 s
TTFT
19.24 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · Ollama
Other
2026-10-19
20.2 tok/s
Decode
85.22 tok/s
Prefill
2.49 s
TTFT
24.78 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · llama.cpp
Other
2026-10-19
67.49 tok/s
Decode
263.38 tok/s
Prefill
2.02 s
TTFT
18.73 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
2026-10-19
66.82 tok/s
Decode
219.12 tok/s
Prefill
2.3 s
TTFT
15.2 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
2026-10-19
43.37 tok/s
Decode
143.55 tok/s
Prefill
2.23 s
TTFT
25.54 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Ultra · 512GB · llama.cpp
Other
2026-10-19
105.53 tok/s
Decode
434.57 tok/s
Prefill
1.74 s
TTFT
17.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
134 results