gemma-4-12B-it

12BChatby Google Start a discussion

0

Quantized builds

139

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

50 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
NVIDIA RTX 4090llama.cppQ3_K_L114.7 tok/s20.24 GB1L2 Cross-framework verified
NVIDIA RTX 4090OllamaQ4_K_M87.4 tok/s22.76 GB1L2 Cross-framework verified
NVIDIA RTX 4090OllamaQ6_K70.2 tok/s22.8 GB2L2 Cross-framework verified
NVIDIA RTX 3090llama.cppQ4_K_M79.3 tok/s15.28 GB2L2 Cross-framework verified
NVIDIA RTX 3090OllamaQ3_K_L64.1 tok/s20.24 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXvLLMQ3_K_M77.9 tok/s19.24 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXOllamaIQ1_S92 tok/s12.68 GB1L2 Cross-framework verified
NVIDIA RTX 5090llama.cppIQ2_XS262.1 tok/s14.2 GB2L2 Cross-framework verified
NVIDIA RTX 5090llama.cppQ8_0125.5 tok/s21.95 GB2L2 Cross-framework verified
NVIDIA RTX 5090OllamaQ3_K_S236.1 tok/s18.48 GB1L2 Cross-framework verified
NVIDIA RTX 5090OllamaQ5_K_S165.1 tok/s24.78 GB1L2 Cross-framework verified
NVIDIA RTX 3060 12GBStrataIQ1_S99.1 tok/s11.4 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppIQ2_M32.6 tok/s15.2 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppQ5_K_M22.7 tok/s25.54 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaQ3_K_M26.8 tok/s19.24 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaQ5_K_S20.2 tok/s24.78 GB1L2 Cross-framework verified
Apple M5 Maxllama.cppIQ3_S67.5 tok/s18.73 GB1L2 Cross-framework verified
Apple M5 MaxOllamaIQ2_M66.8 tok/s15.2 GB1L2 Cross-framework verified
Apple M5 MaxOllamaQ5_K_M43.4 tok/s25.54 GB1L2 Cross-framework verified
Apple M5 Ultrallama.cppIQ3_XS105.5 tok/s17.72 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ3_K_L86.9 tok/s20.24 GB1L2 Cross-framework verified
Apple M5 UltraOllamaQ5_K_S66.6 tok/s24.78 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3llama.cppQ5_K_M303.3 tok/s25.54 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3llama.cppQ5_K_S283.1 tok/s24.78 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMIQ3_XS302.8 tok/s17.72 GB1L2 Cross-framework verified
NVIDIA H100 80GB HBM3vLLMIQ4_XS358.4 tok/s20.75 GB1L2 Cross-framework verified
NVIDIA H20llama.cppIQ2_XS63.1 tok/s14.2 GB1L2 Cross-framework verified
NVIDIA H20vLLMQ3_K_L49.1 tok/s20.24 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBOllamaIQ3_XS204.7 tok/s17.72 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBOllamaQ4_K_M162.2 tok/s22.76 GB1L2 Cross-framework verified
NVIDIA RTX PRO 6000 Blackwell 96GBStrataQ4_K_S255.5 tok/s21.76 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppIQ4_XS90.7 tok/s20.75 GB1L2 Cross-framework verified
Apple M2 Ultrallama.cppQ5_K_S66.9 tok/s24.78 GB1L2 Cross-framework verified
Apple M2 UltraOllamaQ2_K96.5 tok/s15.96 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ3_K_L43.9 tok/s20.24 GB1L2 Cross-framework verified
NVIDIA RTX Sparkllama.cppQ4_K_M39.6 tok/s22.76 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaFP1618.6 tok/s48.72 GB1L2 Cross-framework verified
NVIDIA RTX SparkOllamaQ4_K_M36.6 tok/s22.76 GB1L2 Cross-framework verified
NVIDIA RTX 4090llama.cppIQ2_XS152.3 tok/s14.2 GB1L2 Cross-framework verified
NVIDIA RTX 4090StrataQ5_K_S117.5 tok/s22.8 GB1L2 Cross-framework verified
NVIDIA RTX 3090vLLMQ6_K50.6 tok/s22.8 GB1L2 Cross-framework verified
NVIDIA RTX 3090OllamaIQ3_XS67.3 tok/s17.72 GB1L2 Cross-framework verified
NVIDIA RTX 4060 Ti 16GBvLLMQ3_K_S104.2 tok/s15.2 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXllama.cppIQ2_XS97.1 tok/s14.2 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXllama.cppQ5_K_M54.1 tok/s22.8 GB1L2 Cross-framework verified
AMD Radeon RX 7900 XTXStrataIQ2_M125.3 tok/s15.2 GB1L2 Cross-framework verified
NVIDIA RTX 5090vLLMIQ3_S261.3 tok/s18.73 GB1L2 Cross-framework verified
NVIDIA RTX 5090vLLMQ5_K_S173.7 tok/s24.78 GB1L2 Cross-framework verified
NVIDIA RTX 5090StrataQ3_K_M340.9 tok/s19.24 GB1L2 Cross-framework verified
NVIDIA RTX 5080llama.cppIQ2_M201 tok/s15.2 GB1L2 Cross-framework verified

139 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion