llama.cpp b12059 improves unified KV for multiple sequences, not every GPU workload
The new build includes compact KV row indexing for multi-sequence attention. Earlier upstream measurements are not fresh independent benchmarks or a general speedup promise.
llama.cpp released b12059 within this reporting window, at 22:41 UTC on November 2, 2026. It incorporates changes to attention over a unified KV cache when serving multiple sequences. This is an engine release, not a model launch or an independent hardware reproduction by AraoAI.
-
The problem: with multiple sequences sharing a unified cache, attention must exclude tokens belonging to other sequences. The upstream PR describes an extension, flash_attn_ext_rows, and a kv_rows structure that compactly represents relevant cache rows rather than relying on filtering unrelated positions.
-
The scope: the release notes discuss per-sequence cell lists, omitting cells hidden by sliding-window attention, and handling uneven or interleaved microbatches. These changes are relevant to multi-session local servers. They do not establish the same benefit for a single short desktop chat request, nor do they prove every backend has been validated.
-
The evidence boundary: the linked PR contains earlier developer measurements. Merging it today does not turn those tables into new measurements from this window. We do not reproduce those speeds as fresh community evidence or claim independent validation.
Keep the old build for rollback. Compare both builds on identical weights, KV precision, context, concurrency and prompts. Record time to first token, prefill throughput and per-stream decode separately from aggregate throughput. What is verified here is inclusion in the new release; the benefit on a particular machine still needs a backend-specific measurement.
Sources: b12059 release; upstream PR #29510.