[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f25gh0qpdqnjx0":3},{"slug":4,"category":5,"publishedAt":6,"titleZh":7,"titleEn":8,"summaryZh":9,"summaryEn":10,"models":11,"hardwares":12,"bodyZh":13,"bodyEn":14,"sourceName":15,"sourceUrl":16,"updatedAt":17},"llamacpp-b12008-gemma-tied-embedding-memory-2026-10-31","RUNTIME","2026-10-30T23:08:05.970Z","llama.cpp 修复 Gemma 重复嵌入分配：省的是模型缓冲，不是所有场景的显存","llama.cpp fixes duplicate Gemma embeddings in specific loading paths","b12008 收录一项 Gemma 加载修复：在无 mmap、输入输出同缓冲类型时避免重复分配嵌入表。作者给出 CPU 模型缓冲对照，但不能据此声称全量 GPU 部署提速或省显存。","Build b12008 includes a fix for duplicate tied embeddings in Gemma loading without mmap when input and output share a buffer type. The reported CPU-buffer saving is not a universal GPU-memory or throughput improvement.",[],[],"内存省了，不等于显存都省了。10月30日，llama.cpp 合并 PR #31394，随后发布的 b12008 收录了这项针对 gemma3n、gemma4 的加载修复。它处理的是特定路径下重复分配 token_embd，不是新的量化方法，也不是全模型通用提速。\n\n触发条件有两个：不使用 mmap，以及输入层、输出层使用相同的缓冲类型。原文列出的情况包括 CPU-only、-ngl 0，或输出层留在 CPU 的部分卸载。此前这两个架构先创建 tied output，再创建 token_embd；查找可复用张量时，后者还不存在，于是分配第二份。修复只是把创建顺序调回来。\n\n作者用 gemma-4-E2B-it Q4_K_M、CPU-only、无 mmap 做对照，CPU model buffer 从 3163.73 MiB 降至 2947.73 MiB，原文说明差额为 216 MiB，正好是一份嵌入表。这是提交者自报的特定案例，不是 AraoAI 独立复现，也不是进程总内存或显存峰值测量。作者另称，同模型 CPU-only 的贪心输出在启用和禁用 mmap 时均与原主干一致。\n\n边界同样明确：默认 mmap 下，两份张量本来就指向同一组映射页；-ngl 99 时，输出层位于 GPU，仍按原设计保留自己的副本，作者明确说这一场景不变。因此不能把这项修复改写成“Gemma 全量 GPU 部署节省 216 MiB”，更不能从缓冲大小推导 token\u002Fs。\n\n排查时先记录引擎构建、加载方式与输出层落点，再对比同模型、同量化、同参数的加载日志。把 CPU model buffer、进程 RSS、GPU 占用分别记账，才能知道到底省在哪里。该 PR 还注明架构测试当时跳过 GEMMA4，不能把单例输出一致当成完整回归保证。\n\n来源：PR https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F31394 ；版本 https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12008 。AI 辅助整理，未作独立真机复现。","A smaller model buffer is not automatically a GPU-memory saving. On October 30, llama.cpp merged PR #31394; release b12008 subsequently included the change for gemma3n and gemma4. It removes a duplicate token_embd allocation in a specific loading path, rather than introducing a new quantization method or a general throughput improvement.\n\nThe path requires both loading without mmap and placing the input and output layers in the same buffer type. The author lists CPU-only loading, -ngl 0, and partial offload with the output layer on the CPU. Previously these architectures created the tied output before token_embd. The reuse lookup could not find a tensor that did not exist yet, so a second allocation was made. The fix changes the creation order.\n\nFor gemma-4-E2B-it Q4_K_M, CPU-only and without mmap, the author reports CPU model buffers of 3163.73 MiB before and 2947.73 MiB after, identifying the 216 MiB difference as one embedding table. This is a contributor-reported case, not an independent AraoAI reproduction, process-RSS measurement or GPU-memory peak. The author also reports identical greedy output against the previous master on CPU, with and without mmap.\n\nThe boundaries matter. With default mmap, the tensors already reference the same mapped pages. With -ngl 99, the output is on the GPU and keeps its own embedding copy; the author explicitly says this case is unchanged. Neither a universal GPU saving nor a token\u002Fs gain follows from the buffer comparison.\n\nFor diagnosis, pin the build, loading mode and output-layer placement, then compare logs with the same model, quantization and arguments. Track CPU model buffers, process RSS and GPU allocation separately. The PR also notes that the architecture test skipped GEMMA4 at that time, so the single output check is not a complete regression guarantee.\n\nSources: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F31394 and https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12008 . AI-assisted editorial summary; no independent hardware reproduction.","llama.cpp GitHub PR \u002F release","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F31394","2026-10-30T23:08:05.971Z"]