llama.cpp fixes missing Qwen2.5-VL NVFP4 scales: loading is not correctness
Build b12385 includes an FFN-scale fix. The contributor reports invalid outputs in the old graph. AraoAI has not reproduced it; perplexity diagnostics are not a general ability score.
Source: llama.cpp PR与构建发布说明
On November 14, 2026, llama.cpp merged PR #32241 at 18:02 UTC and included it in build b12385, released at 18:48 UTC. This is not a claim that quantization merely makes a model slightly less capable. The Qwen2-VL graph had failed to apply NVFP4 per-tensor scales that the loader already loaded, allowing a model to start but produce invalid output. These are upstream changes and contributor-reported tests, not an AraoAI reproduction.
The fix passes ffn_up_s, ffn_gate_s and ffn_down_s to build_ffn instead of NULL. Models without these scales retain null values. The PR changes and tests qwen2vl only; it does not establish that other architectures with similar patterns are fixed or that all Qwen vision models gain performance.
On GB10, the contributor converted nvidia/Qwen2.5-VL-7B-Instruct-NVFP4 and tested wikitext-2 with context 512, batch and ubatch 512, five chunks and -fit off. Against baseline 055c3c26e, reported CUDA perplexity changed from 1,043,694 to 9.3759; CPU changed from NaN in every chunk to 9.1177. This is a targeted numerical diagnostic, not an ability leaderboard or a throughput improvement.
For deployment debugging, correctness comes before optimization. Preserve weight hashes, conversion commands, engine commits, backend settings and a fixed input before changing GPU placement or blaming the precision format. After upgrading, check text, image and real task outputs separately. A fix appearing in a build does not validate every backend or quantization. Pin the actual binary, not “latest,” and do not translate recovered perplexity into a promise of lossless capability.
Source: https://github.com/ggml-org/llama.cpp/pull/32241 Release: https://github.com/ggml-org/llama.cpp/releases/tag/b12385