[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$fovu8swugo2he":3},{"slug":4,"category":5,"publishedAt":6,"titleZh":7,"titleEn":8,"summaryZh":9,"summaryEn":10,"models":11,"hardwares":12,"bodyZh":13,"bodyEn":14,"sourceName":15,"sourceUrl":16,"updatedAt":17},"llamacpp-b12121-quantized-kv-state-copy-fix-2026-11-06","RUNTIME","2026-11-05T23:04:09.652Z","llama.cpp 修复量化 KV 状态拷贝：能生成，不代表恢复会话就正确","llama.cpp fixes quantized KV state copies: generation alone does not validate session restore","11月5日合并的修复进入 b12121：设备侧状态保存、恢复及跨分块复制曾把量化块数当元素数。影响特定状态拷贝路径，不能扩大成所有量化 KV 或所有推理请求均有问题。","Merged November 5 and included in b12121, the fix corrects block-versus-element counts in device-side sequence state copies. This concerns a specific save\u002Frestore path, not every quantized KV workload.",[],[],"llama.cpp 于 11 月 5 日 02:31 UTC 合并 #30541，并在当日 03:00 UTC 发布的 b12121 中纳入修复。事件是旧 PR 在本次窗口正式合并与发版，不是今天首次发现问题。对需要保存、恢复或复制会话状态的本地服务，这比单看生成速度更值得检查。\n\n1. 错误发生在特定拷贝路径。PR 说明，启用 LLAMA_STATE_SEQ_FLAGS_ON_DEVICE 时，序列状态通过设备侧 KV 范围副本保存。旧实现用字节数除以元素大小计算拷贝数量；面对 q8_0、q4_0 这样的分块量化类型，得到的却是块数。作者指出，这会只复制部分范围，导致恢复状态错误或拷贝崩溃。不能据此推断所有使用量化 KV 的普通生成请求都会出错。\n\n2. 修的是数量计算，不是宣布量化无损。新实现按类型块大小换算回元素数，同时要求范围覆盖完整量化块；保存、恢复及不同分块范围之间的复制都作了调整。PR 明确 f16、f32 缓存不受这项问题影响。模型权重量化与 KV 量化也是两层配置，不能因为都写着 q4 就混成一件事。\n\n3. 测试证据有范围。提交者报告，CPU 后端的 test-save-load-state 在 q8_0、q4_0 下，旧版设备序列复制测试失败或崩溃，修复后通过；另检查了配合相关修复的 OpenCL 后端。这是上游测试说明，不是 AraoAI 在全部显卡上的独立复现，也不是速度收益。\n\n部署验收应把连续生成与状态恢复分开：固定模型、构建号、后端和 KV 类型，分别检查保存前后的输出一致性、恢复错误及服务存活。只测一次聊天能回答，证明不了会话状态路径正确。现有环境先留旧构建与日志，再针对实际用到的状态功能回归。\n\n来源：[修复 PR](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30541)；[b12121 发布页](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12121)。","llama.cpp merged #30541 at 02:31 UTC on November 5. The fix appears in b12121, published at 03:00 UTC that day. The new event is the merge and release of an older proposal, not a claim that the issue was first discovered today. For a local service that saves, restores or copies conversation state, this deserves a check separate from generation speed.\n\n1. The failure concerns a specific copy path. With LLAMA_STATE_SEQ_FLAGS_ON_DEVICE, sequence state uses device-side copies of KV cache ranges. The old byte-count calculation returned block counts for quantized types such as q8_0 and q4_0 rather than element counts. According to the author, only part of each range was copied, producing incorrect restored state or a crash. This does not establish that every ordinary generation request using quantized KV is broken.\n\n2. The change corrects copy counts, not quantization quality. It converts type blocks back into elements and asserts whole-block ranges in save, restore and differently chunked copy operations. The proposal explicitly says f16 and f32 caches are unaffected. Weight quantization and KV quantization are distinct settings.\n\n3. Evidence has limits. The author reports CPU test-save-load-state failures or crashes with q8_0 and q4_0 before the change and passes afterward, plus an OpenCL check alongside a related fix. These are upstream test reports, not independent AraoAI replication across all GPUs and not a speed claim.\n\nPin the model, build, backend and KV types, then test continuous generation and state restoration separately. Check output consistency, restore errors and process survival. A successful chat response does not validate the session-state path. Preserve the old binary and logs before regression testing the state features actually used.\n\nSources: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30541 ; https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Freleases\u002Ftag\u002Fb12121","llama.cpp GitHub PR \u002F Release","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fpull\u002F30541","2026-11-05T23:04:09.653Z"]