[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f1w3k0vl3lvg9v":3},{"slug":4,"category":5,"publishedAt":6,"titleZh":7,"titleEn":8,"summaryZh":9,"summaryEn":10,"models":11,"hardwares":12,"bodyZh":13,"bodyEn":14,"sourceName":15,"sourceUrl":16,"updatedAt":17},"llamacpp-b12191-moe-cache-cpu-share-2026-11-08","RUNTIME","2026-11-07T23:05:52.365Z","llama.cpp 为 MoE 缓存增加 CPU 分担：少传专家，不等于固定倍速","llama.cpp adds CPU sharing for MoE cache misses, not a universal speed multiplier","11月7日合并的 #30966 进入 b12191：未命中的专家可部分交给 CPU 计算，并新增本机调参脚本。校准结果与预测表必须分开，不能把一台机器的建议参数当通用配置。","Merged November 7 and included in b12191, #30966 can compute a share of missed experts on the CPU and adds a local tuning script. Calibration measurements and predicted rates are separate evidence.",[],[],"llama.cpp 于 11 月 7 日合并 #30966，当日发布的 b12191 纳入这项 MoE 缓存更新。改变的不是模型参数，而是专家缓存未命中后，让 CPU 参与计算的路径。这是本次窗口内的合并与发版，不是把旧 PR 的创建日期当新品日期。\n\n1. 缓存未命中，不必全部上传。新增的 --moe-cache-cpu-share 控制由 CPU 从主机专家权重计算的未命中份额，其余部分仍走设备缓存路径，再合并结果。它把问题从“显存能装多少”推进到“上传与 CPU 计算怎样分工”。不能据此说整模型已常驻显存，也不能把该参数理解成固定的显存占用比例。\n\n2. 调参要在自己的机器上做。新增 scripts\u002Fmoe-cache-tune.py 包装 llama-server 命令，在两种缓存规模和三种 CPU 分担设置下校准，再回放路由、预测其他组合。输出中的校准运行与 Predicted 表不是同一种证据，推荐速度不能当成已经测得的速度写入记录库。不同 CPU、主机内存带宽和设备上传链路，都可能改变分工结果。\n\n3. 上游示例不是选卡结论。PR 给出开发者的一组调参输出，但未形成跨硬件、同协议的独立复现。本文不引用其中速度作显卡排名，也不承诺 CPU 分担越高越快。缓存规模与分担设置必须联合测试，参数名称本身不能证明收益。\n\n部署时先保留旧构建与完整启动参数，再固定模型量化、上下文及请求负载，对照生成速度、输出正确性和内存状态。项目采用 MIT 许可；这不自动覆盖另行下载的模型权重许可。当前可确认的是上游增加了可调路径，不是所有本地 MoE 工作负载都会提速。调参运行也有耗时成本；短请求、长请求及暖缓存状态不同，不能把一次校准结果扩展成全部业务结论。\n\n来源：[修复与功能 PR](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30966)；[b12191 发布页](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12191)。","llama.cpp merged #30966 on November 7 and included the MoE cache update in b12191, released the same day. The change concerns how missed experts are computed, not the model's parameter count. The event within this reporting window is the merge and release of an older proposal.\n\n1. A cache miss need not always trigger an upload. The new --moe-cache-cpu-share setting computes a share of missed experts on the CPU using host weights. The remaining work uses device cache slots, and the results are combined. This makes upload-versus-CPU work a tuning decision. It does not mean the entire model resides in VRAM, and the setting is not a fixed fraction of VRAM usage.\n\n2. Tune on the actual machine. scripts\u002Fmoe-cache-tune.py wraps a llama-server command, calibrates two cache sizes and three CPU shares, then replays routing to predict other combinations. Its calibration runs and Predicted table are different types of evidence. A recommended rate is not a measured result suitable for a benchmark record. CPU capability, host memory bandwidth and the upload link can all change the useful split.\n\n3. The upstream example is not a buying guide. The PR includes one developer's tuning output, not independent, protocol-matched evidence across hardware. We do not turn its speed table into a GPU ranking or claim that a higher CPU share is always better. Cache size and CPU share need to be tested together.\n\nKeep the previous build and complete launch arguments. Hold model quantization, context and workload constant while comparing generation speed, output correctness and memory state. llama.cpp uses the MIT license; model weights have separate terms. The verified news is a new configurable execution path, not a universal speed improvement. Calibration also costs time. Short and long requests and different cache states need their own checks; one calibration does not establish performance for every workload.\n\nSources: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30966 and https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12191.","llama.cpp GitHub","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30966","2026-11-07T23:05:52.366Z"]