llama.cpp b12611 fixes Inkling 2 loop scales: loading is not correctness
PR #32704, merged November 21, re-shares loop slots after scale tensors are attached. It targets the scaled NVFP4 path, not every GGUF. AraoAI has not independently reproduced it.
Source: llama.cpp GitHub PR与发布说明
On November 21, llama.cpp merged PR #32704 and subsequently published release b12611 containing the fix. This is an Inkling 2 loading-correctness change, not a claim about fitting larger contexts or generating faster. A model may start and return readable text while executing part of its computation incorrectly.
-
Shared loop slots must include scales. According to the PR, inkling2_loop previously copied physical layers into their loop slots before the generic loading pass attached optional .scale tensors. The additional slots therefore lacked the per-expert scales needed on the second and third passes through the core layers. The fix re-shares the slots after the scale pass.
-
Do not generalize the scope to every GGUF. The author explicitly distinguishes ordinary ggml quantizations, which are unaffected by this issue. Follow-up comments report an unchanged IQ4_NL result and separate before/after results for a Mainline_Support UD-IQ1_S file. Those are different weight files, not a single interchangeable comparison. Perplexity is a numerical diagnostic, not a universal measure of retained task capability.
-
Correctness and memory failures are separate. A participant still reports Vulkan compute-buffer OOMs and a smaller microbatch workaround. That is one community report, not a universal recommendation or proof that every platform now works. Memory admission, backend support and output correctness require separate checks.
AraoAI checked the PR, its in-window corrections, merge timestamp and b12611 release notes, but has not reproduced the change on hardware. Preserve the engine build, exact weight file and publisher when troubleshooting. Verify that the actual binary contains the fix before comparing identical inputs; one successful chat is not a correctness evaluation.
Source: https://github.com/ggml-org/llama.cpp/pull/32704 Release: https://github.com/ggml-org/llama.cpp/releases/tag/b12611