News
What is changing in local AI — model releases, hardware news, runtime updates, and deployment guides.
News
New dual-3090 QEngine report separates short prompts, deep context and whole replies
A November 19 report covers QEngine 1.3.6-vision on two 3090s capped at 260 W each. Paired 10K/90K measurements are short-output probes, while whole-reply TTFT includes generation. This remains a community self-report.
llama.cpp fixes Metal private-buffer ranges on discrete-GPU Macs and opt-in paths
Merged November 19 and shipped in b12548, #32602 fixes the length argument to NSMakeRange on the private-buffer path. It is not a general Apple Silicon speedup; upstream says this path is outside CI.
Arc Pro B65 report isolates a long-prefill regression to Intel runtime packages
One community report holds llama.cpp a956a852 and Qwen3-Coder-30B-A3B-Instruct Q4_K_M constant while changing Intel packages. Long prefill falls while decode rises. No independent reproduction supports generalizing to other GPUs.
llama.cpp b12443 fixes NVIDIA Vulkan FP32 Flash Attention shared-memory overflow
Merged on November 16, the fix disables shared-memory staging when its requirement exceeds device capacity. It targets a particular Vulkan Flash Attention path, not model VRAM capacity or a universal NVIDIA speedup.
Four RTX 5080 power-cap sweeps: distinguish the configured limit from actual draw
A QEngine 1.3.5.9 community report includes per-card sweeps and explicit corrections. Higher caps did not produce a consistent throughput gain on this rig; it does not establish universal RTX 5080 savings.
Windows AMD Vulkan regression reports: an RX 7800 XT workaround does not transfer to RX 9070 XT
Two community reports describe Flash Attention crashes with the proprietary AMD Vulkan driver on Windows. Workarounds differ between the GPUs; this is not evidence of a universal AMD failure.
A four-5080 Flash-Next report: 448.60 tok/s needs power and output conditions
A QEngine 1.3.5.9 community report states power caps, host offload and whole-reply timing limits. Weight quantization is not explicit in the retrieved report, so no benchmark row is imported.
llama.cpp fixes missing Qwen2.5-VL NVFP4 scales: loading is not correctness
Build b12385 includes an FFN-scale fix. The contributor reports invalid outputs in the old graph. AraoAI has not reproduced it; perplexity diagnostics are not a general ability score.
A three-3090 Flash-Next report: 334.86 tok/s needs its ESXi, patch and MTP conditions
A new community report records QEngine 1.3.5.7 on three 3090s. Power caps, missing P2P, a tensor patch and benchmark phases are essential conditions, not optional footnotes. No independent AraoAI reproduction is claimed.
llama.cpp v1.0.0: check API and server boundaries before treating the release as a speed upgrade
The release removes deprecated APIs, adds FP8 KV caches and MoE host-cache controls, and tightens server security. These are release facts, not an AraoAI performance reproduction.
89 articles