[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f3cpmfoz64rehm":3},{"slug":4,"category":5,"publishedAt":6,"titleZh":7,"titleEn":8,"summaryZh":9,"summaryEn":10,"models":11,"hardwares":12,"bodyZh":13,"bodyEn":14,"sourceName":15,"sourceUrl":16,"updatedAt":17},"llamacpp-b12328-moe-cache-stale-prefetch-2026-11-13","RUNTIME","2026-11-12T23:12:30.579Z","llama.cpp 修复 MoE 缓存预取次序：先检查结果正确，再比较速度","llama.cpp fixes MoE cache prefetch ordering: verify correctness before speed","11月12日合并的#32100进入b12328，避免部分跨后端调度提前读到旧专家权重，并在设备不支持缓存算子时回退。修复针对特定路径，不是所有MoE模型的通用加速。","PR #32100 merged on November 12 and shipped in b12328. It prevents stale expert reads in a cross-backend scheduling path and bypasses unsupported cache operations. This is a targeted correctness fix, not a universal MoE speedup.",[],[],"11月12日，llama.cpp 合并 PR #32100，并在当日发布的 b12328 中列出该修复。问题出在 MoE 缓存与跨后端调度的交界：下一段计算的输入可能在专家上传完成前被预取，随后使用旧的专家内容。这不是“速度看起来慢一点”，而是结果正确性需要先过关。\n\n1. 先分清触发条件。维护者说明，缓存设备不支持对应的 MUL_MAT_ID 算子时，调度器可能把矩阵运算安排到CPU，设备上的专家缓存便成为CPU计算段的输入。上传由复制回调完成，而原先预取发生得更早；提前读取就可能拿到尚未更新的内容。部分Vulkan路径是文中指出的场景，不等于所有Vulkan设备都受影响。\n\n2. 修复同时处理顺序与能力检查。新逻辑只预取计算缓冲区里的输入，即前序计算段产生的结果；建立缓存槽位时还检查设备能否执行缓存算子，不能就走不缓存路径，避免每个微批都把整块缓存搬往另一后端。不能把“不用这条缓存路径”误读成模型被裁剪或量化精度降低。\n\n3. 原文测试证明的边界要保留。PR提供了调度器回归测试与架构测试，也说明普通CUDA缓存路径的输入仍为计算得到的激活，行为不变。这是提交者的开发验证，不是本站在多种硬件上的独立复现；本文不将局部测试改写为整模型提速百分比。\n\n本地排查时，先保存实际构建号、后端、缓存设置、权重身份和出错输入，再在固定配置下对照升级前后的输出。确认正确后，才分别比较首字延迟、prefill、decode与搬运开销。能生成流畅文字不证明专家数据没读错，跑得更快也不能替代正确性验收。\n\n来源：https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F32100\n发布：https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12328","On November 12, llama.cpp merged PR #32100 and included it in release b12328. The issue sits between MoE caching and cross-backend scheduling: an input for the next compute split could be prefetched before a copy callback uploaded the new experts. The following split could therefore read stale weights. Correctness comes before throughput in this case.\n\n1. The trigger is specific. If the cache device cannot execute the cached MUL_MAT_ID operation, the scheduler may place the matrix multiplication on the CPU. The device bank then becomes an input to a CPU split. Prefetching that bank before the upload callback reads it too early. The author identifies some Vulkan cases, not every Vulkan device.\n\n2. The fix changes ordering and capability checks. Prefetch now targets inputs in compute buffers, meaning results produced by earlier splits. Cache-slot construction also checks whether the selected device supports the cached operation, using the uncached path otherwise. This avoids transferring the whole bank to another backend for each micro-batch. Bypassing this cache is not model pruning or reduced weight precision.\n\n3. Preserve the test boundary. The PR supplies scheduler regression and architecture tests and states that the usual CUDA cache path is unchanged because its prefetched inputs are computed activations. These are developer tests, not independent multi-device reproduction by AraoAI. They do not establish an end-to-end speedup percentage.\n\nFor deployment debugging, preserve the actual build, backend, cache settings, weight identity and failing input. Compare outputs under a fixed configuration before comparing first-token latency, prefill, decode or transfers. Fluent text is not proof that the correct expert data was read. Faster execution cannot replace a correctness check.\n\nSource: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F32100\nRelease: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12328","llama.cpp GitHub PR \u002F release","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F32100","2026-11-12T23:12:30.580Z"]