gemma-4-31B-it

31BChatby Google Start a discussion

0

Quantized builds

75

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

50 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX Sparkllama.cppFP164.7 tok/s110.2 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ6_K6.4 tok/s67.2 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ8_05.8 tok/s76.97 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaFP164 tok/s110.2 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ8_05.3 tok/s76.97 GB1L2 Cross-framework verified
NVIDIA RTX 5090StrataIQ2_M284.3 tok/s30.4 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppIQ2_M24.4 tok/s35.95 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppIQ2_XS25.1 tok/s33.35 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaIQ2_M24.9 tok/s35.95 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppIQ1_S64.7 tok/s29.44 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppIQ2_XS58.2 tok/s33.35 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ4_K_M32.9 tok/s55.48 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppQ3_K_L65.9 tok/s48.97 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppQ8_038 tok/s76.97 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ2_K75.9 tok/s37.91 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ8_038.9 tok/s76.97 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMIQ3_XS338.7 tok/s42.46 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMQ3_K_M275.7 tok/s46.37 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3StrataQ5_K_S299.9 tok/s60.69 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3StrataQ6_K276.8 tok/s67.2 GB1L2 Cross-framework verified
NVIDIA H20vLLMIQ3_XS40 tok/s42.46 GB1L2 Cross-framework verified
NVIDIA H20vLLMQ4_K_S32.6 tok/s52.88 GB1L2 Cross-framework verified
NVIDIA H20OllamaQ8_019 tok/s76.97 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBvLLMIQ2_XS189.8 tok/s33.35 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBOllamaIQ1_S184.4 tok/s29.44 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBOllamaQ4_K_S134.3 tok/s52.88 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppQ4_K_M59 tok/s55.48 GB1L2 Cross-framework verified
Apple M2 UltraOllamaIQ2_XS73.2 tok/s33.35 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ4_K_M8.8 tok/s55.48 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ3_K_S10.2 tok/s44.41 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ4_K_M8 tok/s55.48 GB1L2 Cross-framework verified
NVIDIA RTX 5090vLLMIQ2_M219.7 tok/s30.4 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395StrataIQ3_XS81.3 tok/s42.46 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395StrataQ3_K_S82.3 tok/s44.41 GB1L2 Cross-framework verified
Apple M4 ProvLLMIQ2_M64.9 tok/s35.95 GB1L2 Cross-framework verified
Apple M5 ProOllamaQ3_K_M62.3 tok/s46.37 GB1L2 Cross-framework verified
Apple M5 ProStrataIQ3_XS111.4 tok/s42.46 GB1L2 Cross-framework verified
Apple M5 MaxStrataIQ4_XS138.4 tok/s50.27 GB1L2 Cross-framework verified
Apple M5 MaxStrataQ3_K_M149.4 tok/s46.37 GB1L2 Cross-framework verified
NVIDIA DGX Sparkllama.cppIQ3_XS91.6 tok/s42.46 GB1L2 Cross-framework verified
NVIDIA DGX SparkOllamaQ6_K50.1 tok/s67.2 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBOllamaIQ3_XS29.4 tok/s42.46 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBOllamaQ3_K_M26.6 tok/s46.37 GB1L2 Cross-framework verified
Apple M3 Maxllama.cppQ2_K64.5 tok/s34.2 GB1L2 Cross-framework verified
Apple M3 MaxOllamaQ2_K65.7 tok/s34.2 GB1L2 Cross-framework verified
NVIDIA RTX 5090llama.cppIQ2_M159.3 tok/s30.4 GB1L2 Cross-framework verified
NVIDIA RTX 5090OllamaIQ1_S146.2 tok/s29.44 GB1L2 Cross-framework verified
Apple M4 Maxllama.cppQ3_K_M93 tok/s46.37 GB1L2 Cross-framework verified
Apple M4 Maxllama.cppQ8_053.1 tok/s76.97 GB1L2 Cross-framework verified
Apple M4 MaxvLLMQ2_K111.5 tok/s37.91 GB1L2 Cross-framework verified

75 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion