Benchmark search

Try these searches

Found 134 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

Modelgemma-4-12B-it

gemma-4-12B-it · Q3_K_L

L2 Cross-framework verified

NVIDIA RTX 4090 · 24GB · llama.cpp

Other

2026-10-19

114.65 tok/s

Decode

473.3 tok/s

Prefill

2.16 s

TTFT

20.24 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.4 Q3_K_L llama.cpp v0.23 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q4_K_M

L2 Cross-framework verified

NVIDIA RTX 4090 · 24GB · Ollama

Other

2026-10-19

87.43 tok/s

Decode

399.69 tok/s

Prefill

1.92 s

TTFT

22.76 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 Q4_K_M Ollama v0.31 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q6_K

L2 Cross-framework verified

NVIDIA RTX 4090 · 24GB · Ollama

Other

2026-10-19

70.15 tok/s

Decode

334.66 tok/s

Prefill

1.31 s

TTFT

22.8 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q6_K Ollama v0.35

Sources: Other · 2 runs · 1 independent sources

gemma-4-12B-it · Q4_K_M

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · llama.cpp

Other

2026-10-19

79.26 tok/s

Decode

473.41 tok/s

Prefill

1.21 s

TTFT

15.28 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 Q4_K_M llama.cpp v0.50 Flash Attention

Sources: Other · 2 runs · 1 independent sources

gemma-4-12B-it · Q3_K_L

L2 Cross-framework verified

NVIDIA RTX 3090 · 24GB · Ollama

Other

2026-10-19

64.09 tok/s

Decode

302.97 tok/s

Prefill

1.12 s

TTFT

20.24 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q3_K_L Ollama v0.28 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q3_K_M

L2 Cross-framework verified

AMD Radeon RX 7900 XTX · 24GB · vLLM

Other

2026-10-19

77.87 tok/s

Decode

405.9 tok/s

Prefill

0.18 s

TTFT

19.24 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 Q3_K_M vLLM v0.48 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ1_S

L2 Cross-framework verified

AMD Radeon RX 7900 XTX · 24GB · Ollama

Other

2026-10-19

92.04 tok/s

Decode

422.92 tok/s

Prefill

2.03 s

TTFT

12.68 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.4 IQ1_S Ollama v0.21 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ2_XS

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-19

262.05 tok/s

Decode

997.54 tok/s

Prefill

1.68 s

TTFT

14.2 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.8 IQ2_XS llama.cpp v0.20 Flash Attention

Sources: Other · 2 runs · 1 independent sources

gemma-4-12B-it · Q8_0

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · llama.cpp

Other

2026-10-19

125.52 tok/s

Decode

751.15 tok/s

Prefill

0.88 s

TTFT

21.95 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.6 Q8_0 llama.cpp v0.21 Flash Attention

Sources: Other · 2 runs · 1 independent sources

gemma-4-12B-it · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Ollama

Other

2026-10-19

236.13 tok/s

Decode

819.72 tok/s

Prefill

1.31 s

TTFT

18.48 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.8 Q3_K_S Ollama v0.30

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q5_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Ollama

Other

2026-10-19

165.12 tok/s

Decode

690.64 tok/s

Prefill

1.85 s

TTFT

24.78 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q5_K_S Ollama v0.54 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ1_S

L2 Cross-framework verified

NVIDIA RTX 3060 12GB · 12GB · Strata

Other

2026-10-19

99.13 tok/s

Decode

497.2 tok/s

Prefill

0.34 s

TTFT

11.4 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.8 IQ1_S Strata v0.51 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ2_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

2026-10-19

32.57 tok/s

Decode

131.77 tok/s

Prefill

2.15 s

TTFT

15.2 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.4 IQ2_M llama.cpp v0.24 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q5_K_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

2026-10-19

22.71 tok/s

Decode

109.92 tok/s

Prefill

0.47 s

TTFT

25.54 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q5_K_M llama.cpp v0.27 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q3_K_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · Ollama

Other

2026-10-19

26.82 tok/s

Decode

137.08 tok/s

Prefill

0.81 s

TTFT

19.24 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.4 Q3_K_M Ollama v0.54 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q5_K_S

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · Ollama

Other

2026-10-19

20.2 tok/s

Decode

85.22 tok/s

Prefill

2.49 s

TTFT

24.78 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 Q5_K_S Ollama v0.32 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ3_S

L2 Cross-framework verified

Apple M5 Max · 128GB · llama.cpp

Other

2026-10-19

67.49 tok/s

Decode

263.38 tok/s

Prefill

2.02 s

TTFT

18.73 GB

VRAM

Config (latest run)
macOS 26.1 IQ3_S llama.cpp v0.30 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ2_M

L2 Cross-framework verified

Apple M5 Max · 128GB · Ollama

Other

2026-10-19

66.82 tok/s

Decode

219.12 tok/s

Prefill

2.3 s

TTFT

15.2 GB

VRAM

Config (latest run)
macOS 26.1 IQ2_M Ollama v0.24 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · Q5_K_M

L2 Cross-framework verified

Apple M5 Max · 128GB · Ollama

Other

2026-10-19

43.37 tok/s

Decode

143.55 tok/s

Prefill

2.23 s

TTFT

25.54 GB

VRAM

Config (latest run)
macOS 26.1 Q5_K_M Ollama v0.49 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-12B-it · IQ3_XS

L2 Cross-framework verified

Apple M5 Ultra · 512GB · llama.cpp

Other

2026-10-19

105.53 tok/s

Decode

434.57 tok/s

Prefill

1.74 s

TTFT

17.72 GB

VRAM

Config (latest run)
macOS 26.1 IQ3_XS llama.cpp v0.38

Sources: Other · 1 runs · 1 independent sources

134 results

Related models
Related hardware
Related frameworks
Related quantizations