llama.cpp b12443 fixes NVIDIA Vulkan FP32 Flash Attention shared-memory overflow
Merged on November 16, the fix disables shared-memory staging when its requirement exceeds device capacity. It targets a particular Vulkan Flash Attention path, not model VRAM capacity or a universal NVIDIA speedup.
Source: llama.cpp GitHub PR与发布记录
On November 16, llama.cpp merged PR #32244 and included it in release b12443. The change disables shared-memory staging when Flash Attention would require more shared memory than the device provides. The PR specifically mentions NVIDIA devices lacking FP16 capability for this path. This is an upstream code change, not a hardware reproduction by AraoAI; it supplies no general speedup percentage.
Shared memory here is not the same resource as VRAM used for model weights and KV cache. It is local storage used by GPU workgroups. A model can fit in VRAM while a particular kernel still exceeds its shared-memory limit. Choosing a smaller quantization therefore does not necessarily address this failure.
The patch does not increase device capacity. It avoids the staging path when that path needs too much shared memory. This should not be read as evidence that all Flash Attention is broken, or that CUDA has the same problem. The PR provides no cross-GPU results, so we do not invent a list of validated cards.
For troubleshooting, keep the old build and record the GPU, driver, Vulkan capabilities and failure log. Retest with b12443 or a later build containing the fix while holding the model, quantization, context and launch settings constant. First establish whether the failure disappears; only then compare performance. A successful startup is not proof of long-run stability.
At collection time the PR had no public independent reproduction in its comments. The release verifies that the fix shipped, not that every older card has been tested.
Source: https://github.com/ggml-org/llama.cpp/pull/32244 Release: https://github.com/ggml-org/llama.cpp/releases/tag/b12443