gemma-4-26B-A4B-it

26BChatby Google Start a discussion

0

Quantized builds

39

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

Benchmark data

39 configurations

HardwareFrameworkQuantDecodeVRAM usedSamplesEvidence
AMD Ryzen AI Max+ 395llama.cppIQ2_XS92.6 tok/s28.31 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395llama.cppIQ3_XS81.1 tok/s35.95 GB1L2 Cross-framework verified
AMD Ryzen AI Max+ 395OllamaIQ3_XS79.4 tok/s35.95 GB1L2 Cross-framework verified
Apple M4 ProvLLMIQ3_XS87.6 tok/s35.95 GB1L2 Cross-framework verified
Apple M4 ProvLLMQ4_K_S72.5 tok/s44.69 GB1L2 Cross-framework verified
Apple M4 ProOllamaQ5_K_M56.2 tok/s52.88 GB1L2 Cross-framework verified
Apple M4 ProOllamaQ5_K_S59.3 tok/s51.24 GB1L2 Cross-framework verified
Apple M5 Prollama.cppQ8_055.3 tok/s60.8 GB1L2 Cross-framework verified
Apple M5 ProStrataQ6_K98.9 tok/s56.7 GB1L2 Cross-framework verified
Apple M5 MaxvLLMQ3_K_S174.4 tok/s37.59 GB1L2 Cross-framework verified
Apple M5 MaxvLLMQ4_K_S136.4 tok/s44.69 GB1L2 Cross-framework verified
Apple M5 MaxStrataQ2_K229 tok/s32.13 GB1L2 Cross-framework verified
Apple M5 MaxStrataQ8_0122.6 tok/s64.89 GB1L2 Cross-framework verified
NVIDIA DGX Sparkllama.cppIQ3_S112.4 tok/s38.14 GB1L2 Cross-framework verified
NVIDIA DGX Sparkllama.cppQ3_K_L105.6 tok/s41.41 GB1L2 Cross-framework verified
NVIDIA DGX SparkOllamaQ3_K_L104.2 tok/s41.41 GB1L2 Cross-framework verified
NVIDIA DGX SparkOllamaQ3_K_M113.1 tok/s39.23 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMIQ4_XS42.8 tok/s42.5 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBvLLMQ5_K_S34.1 tok/s51.24 GB1L2 Cross-framework verified
NVIDIA Jetson AGX Orin 64GBOllamaQ8_024.7 tok/s60.8 GB1L2 Cross-framework verified
NVIDIA RTX 5070 TivLLMIQ2_XS86.5 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 5070 TivLLMQ5_K_M39.5 tok/s16 GB1L0 Self-reported
NVIDIA RTX 5070 TivLLMQ4_K_M48.2 tok/s16 GB1L0 Self-reported
NVIDIA RTX 5070 Tillama.cppIQ2_XS79.4 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 5070 Tillama.cppQ5_K_M36.2 tok/s16 GB1L0 Self-reported
NVIDIA RTX 5070 Tillama.cppQ4_K_M44.2 tok/s16 GB1L0 Self-reported
NVIDIA RTX 5090vLLMIQ2_XS173.1 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 5090vLLMQ5_K_M78.9 tok/s19.8 GB1L0 Self-reported
NVIDIA RTX 5090vLLMQ4_K_M96.4 tok/s16.3 GB1L0 Self-reported
NVIDIA RTX 5090llama.cppIQ2_XS158.8 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 5090llama.cppQ5_K_M72.4 tok/s19.8 GB1L0 Self-reported
NVIDIA RTX 5090llama.cppQ4_K_M88.5 tok/s16.3 GB1L0 Self-reported
NVIDIA RTX 4090vLLMIQ2_XS97.4 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 4090vLLMQ5_K_M44.4 tok/s19.8 GB1L0 Self-reported
NVIDIA RTX 4090vLLMQ4_K_M54.2 tok/s16.3 GB1L0 Self-reported
NVIDIA RTX 4090llama.cppIQ2_XS89.3 tok/s9.3 GB1L0 Self-reported
NVIDIA RTX 4090llama.cppQ5_K_M40.7 tok/s19.8 GB1L0 Self-reported
NVIDIA RTX 4090llama.cppQ4_K_M49.8 tok/s16.3 GB1L0 Self-reported
Apple M2 Ultrallama.cppQ4_K_M79.1 tok/s—1L1 Reproduced

39 of these records are not linked to a specific quantized build; the full set is on the search page.

Community discussion

No discussions yet — start the first one.

Start a discussion