llama.cpp fixes duplicate Gemma embeddings in specific loading paths
Build b12008 includes a fix for duplicate tied embeddings in Gemma loading without mmap when input and output share a buffer type. The reported CPU-buffer saving is not a universal GPU-memory or throughput improvement.
Source: llama.cpp GitHub PR / release
A smaller model buffer is not automatically a GPU-memory saving. On October 30, llama.cpp merged PR #31394; release b12008 subsequently included the change for gemma3n and gemma4. It removes a duplicate token_embd allocation in a specific loading path, rather than introducing a new quantization method or a general throughput improvement.
The path requires both loading without mmap and placing the input and output layers in the same buffer type. The author lists CPU-only loading, -ngl 0, and partial offload with the output layer on the CPU. Previously these architectures created the tied output before token_embd. The reuse lookup could not find a tensor that did not exist yet, so a second allocation was made. The fix changes the creation order.
For gemma-4-E2B-it Q4_K_M, CPU-only and without mmap, the author reports CPU model buffers of 3163.73 MiB before and 2947.73 MiB after, identifying the 216 MiB difference as one embedding table. This is a contributor-reported case, not an independent AraoAI reproduction, process-RSS measurement or GPU-memory peak. The author also reports identical greedy output against the previous master on CPU, with and without mmap.
The boundaries matter. With default mmap, the tensors already reference the same mapped pages. With -ngl 99, the output is on the GPU and keeps its own embedding copy; the author explicitly says this case is unchanged. Neither a universal GPU saving nor a token/s gain follows from the buffer comparison.
For diagnosis, pin the build, loading mode and output-layer placement, then compare logs with the same model, quantization and arguments. Track CPU model buffers, process RSS and GPU allocation separately. The PR also notes that the architecture test skipped GEMMA4 at that time, so the single output check is not a complete regression guarantee.
Sources: https://github.com/ggml-org/llama.cpp/pull/31394 and https://github.com/ggml-org/llama.cpp/releases/tag/b12008 . AI-assisted editorial summary; no independent hardware reproduction.