Benchmark search

Try these searches

Found 75 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

Modelgemma-4-31B-it

gemma-4-31B-it · FP16

L2 Cross-framework verified

NVIDIA RTX Spark · 128GB · llama.cpp

Other

2026-10-20

4.66 tok/s

Decode

23.51 tok/s

Prefill

3.09 s

TTFT

110.2 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 FP16 llama.cpp v0.27

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q6_K

L2 Cross-framework verified

NVIDIA RTX Spark · 128GB · llama.cpp

Other

2026-10-20

6.37 tok/s

Decode

31.53 tok/s

Prefill

0.54 s

TTFT

67.2 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 Q6_K llama.cpp v0.39 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q8_0

L2 Cross-framework verified

NVIDIA RTX Spark · 128GB · llama.cpp

Other

2026-10-20

5.75 tok/s

Decode

29.94 tok/s

Prefill

1.91 s

TTFT

76.97 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 Q8_0 llama.cpp v0.30

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · FP16

L2 Cross-framework verified

NVIDIA RTX Spark · 128GB · Ollama

Other

2026-10-20

3.97 tok/s

Decode

19.3 tok/s

Prefill

1.68 s

TTFT

110.2 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 FP16 Ollama v0.25

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q8_0

L2 Cross-framework verified

NVIDIA RTX Spark · 128GB · Ollama

Other

2026-10-20

5.28 tok/s

Decode

19.56 tok/s

Prefill

3.29 s

TTFT

76.97 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 Q8_0 Ollama v0.37 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ2_M

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Strata

Other

2026-10-19

284.29 tok/s

Decode

1,098.95 tok/s

Prefill

0.1 s

TTFT

30.4 GB

VRAM

Config (latest run)
Windows 11 24H2 CUDA 12.8 IQ2_M Strata v0.23

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ2_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

2026-10-19

24.41 tok/s

Decode

110.5 tok/s

Prefill

0.82 s

TTFT

35.95 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 IQ2_M llama.cpp v0.53

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ2_XS

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

2026-10-19

25.14 tok/s

Decode

89.05 tok/s

Prefill

0.16 s

TTFT

33.35 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 IQ2_XS llama.cpp v0.43

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ2_M

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · Ollama

Other

2026-10-19

24.86 tok/s

Decode

123.01 tok/s

Prefill

1.69 s

TTFT

35.95 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 IQ2_M Ollama v0.49 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ1_S

L2 Cross-framework verified

Apple M5 Max · 128GB · llama.cpp

Other

2026-10-19

64.71 tok/s

Decode

267.16 tok/s

Prefill

0.33 s

TTFT

29.44 GB

VRAM

Config (latest run)
macOS 26.1 IQ1_S llama.cpp v0.41 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ2_XS

L2 Cross-framework verified

Apple M5 Max · 128GB · llama.cpp

Other

2026-10-19

58.24 tok/s

Decode

303.37 tok/s

Prefill

2.49 s

TTFT

33.35 GB

VRAM

Config (latest run)
macOS 26.1 IQ2_XS llama.cpp v0.53

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q4_K_M

L2 Cross-framework verified

Apple M5 Max · 128GB · Ollama

Other

2026-10-19

32.9 tok/s

Decode

145.14 tok/s

Prefill

1.6 s

TTFT

55.48 GB

VRAM

Config (latest run)
macOS 26.1 Q4_K_M Ollama v0.20

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q3_K_L

L2 Cross-framework verified

Apple M5 Ultra · 512GB · llama.cpp

Other

2026-10-19

65.89 tok/s

Decode

258.39 tok/s

Prefill

2.06 s

TTFT

48.97 GB

VRAM

Config (latest run)
macOS 26.1 Q3_K_L llama.cpp v0.43

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q8_0

L2 Cross-framework verified

Apple M5 Ultra · 512GB · llama.cpp

Other

2026-10-19

37.99 tok/s

Decode

165.65 tok/s

Prefill

2.29 s

TTFT

76.97 GB

VRAM

Config (latest run)
macOS 26.1 Q8_0 llama.cpp v0.38

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q2_K

L2 Cross-framework verified

Apple M5 Ultra · 512GB · Ollama

Other

2026-10-19

75.92 tok/s

Decode

280.65 tok/s

Prefill

1.29 s

TTFT

37.91 GB

VRAM

Config (latest run)
macOS 26.1 Q2_K Ollama v0.42 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q8_0

L2 Cross-framework verified

Apple M5 Ultra · 512GB · Ollama

Other

2026-10-19

38.85 tok/s

Decode

140.28 tok/s

Prefill

2.44 s

TTFT

76.97 GB

VRAM

Config (latest run)
macOS 26.1 Q8_0 Ollama v0.26 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · IQ3_XS

L2 Cross-framework verified

NVIDIA H100 80GB HBM3 · 80GB · vLLM

Other

2026-10-19

338.72 tok/s

Decode

1,535.63 tok/s

Prefill

0.31 s

TTFT

42.46 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.8 IQ3_XS vLLM v0.24 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q3_K_M

L2 Cross-framework verified

NVIDIA H100 80GB HBM3 · 80GB · vLLM

Other

2026-10-19

275.73 tok/s

Decode

1,508 tok/s

Prefill

1.79 s

TTFT

46.37 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.8 Q3_K_M vLLM v0.42 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q5_K_S

L2 Cross-framework verified

NVIDIA H100 80GB HBM3 · 80GB · Strata

Other

2026-10-19

299.9 tok/s

Decode

1,290.71 tok/s

Prefill

2.15 s

TTFT

60.69 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.8 Q5_K_S Strata v0.50 Flash Attention

Sources: Other · 1 runs · 1 independent sources

gemma-4-31B-it · Q6_K

L2 Cross-framework verified

NVIDIA H100 80GB HBM3 · 80GB · Strata

Other

2026-10-19

276.83 tok/s

Decode

1,508.76 tok/s

Prefill

0.3 s

TTFT

67.2 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 13.0 Q6_K Strata v0.53 Flash Attention

Sources: Other · 1 runs · 1 independent sources

75 results

Related models
Related hardware
Related frameworks
Related quantizations