llama.cpp fixes quantized KV state copies: generation alone does not validate session restore

llama.cpp merged #30541 at 02:31 UTC on November 5. The fix appears in b12121, published at 03:00 UTC that day. The new event is the merge and release of an older proposal, not a claim that the issue was first discovered today. For a local service that saves, restores or copies conversation state, this deserves a check separate from generation speed.

1. The failure concerns a specific copy path. With LLAMA_STATE_SEQ_FLAGS_ON_DEVICE, sequence state uses device-side copies of KV cache ranges. The old byte-count calculation returned block counts for quantized types such as q8_0 and q4_0 rather than element counts. According to the author, only part of each range was copied, producing incorrect restored state or a crash. This does not establish that every ordinary generation request using quantized KV is broken.

2. The change corrects copy counts, not quantization quality. It converts type blocks back into elements and asserts whole-block ranges in save, restore and differently chunked copy operations. The proposal explicitly says f16 and f32 caches are unaffected. Weight quantization and KV quantization are distinct settings.

3. Evidence has limits. The author reports CPU test-save-load-state failures or crashes with q8_0 and q4_0 before the change and passes afterward, plus an OpenCL check alongside a related fix. These are upstream test reports, not independent AraoAI replication across all GPUs and not a speed claim.

Pin the model, build, backend and KV types, then test continuous generation and state restoration separately. Check output consistency, restore errors and process survival. A successful chat response does not validate the session-state path. Preserve the old binary and logs before regression testing the state features actually used.

Sources: https://github.com/ggml-org/llama.cpp/pull/30541 ; https://github.com/ggml-org/llama.cpp/releases/tag/b12121