← News
Runtime

New Strix Halo report: explicit mmap can conflict with automatic lazy mode

Issue #31943, filed November 9, reports a same-machine comparison; proposed fix #31957 remains unmerged. Short-prompt prefill and long requests show why local ratios are not overall throughput.

Source: llama.cpp 社区 issue 与待审 PR

Issue #31943 was filed in llama.cpp on November 9. The author reports a mismatch between explicit mmap loading and automatic lazy mode on a 128 GB Strix Halo with Radeon 8060S, RADV/Vulkan, running Qwen3.8-Flash-Next IQ4_NL. This is one reporter's comparison, not AraoAI replication. Fix #31957 remains open and unmerged. Filing time is not a substitute for a stated measurement time.

  1. Auto is not a fixed configuration. The report says a device-capability check disables lazy mode even when mmap is explicitly requested, affecting PLE row prefetch. The proposed patch limits the fallback to cases where mmap is actually off. It is not a released fix or evidence against every integrated GPU.

  2. Compare rows from the same test. On base commit 0c6dca2fa, llama-bench uses pp512, n=0 and three repetitions. mmap/auto reports 247.09±12.71 tok/s, versus 586.17±9.45 for mmap/on. This is short-prompt prefill, not decode. The default auto/auto row is faster still, so manual enablement is not a universal recommendation.

  3. A microbenchmark is not a long request. A single cold 30,187-token server prompt reports 454 versus 493 tok/s for mmap versus mmap/on. That is a different workload with only one run. MemAvailable is host available memory, not measured VRAM use; the cited 95 GiB model size is not VRAM occupancy either. The proposed fix also retains an unusually slow repetition rather than silently discarding it.

For diagnosis, pin the build, loading path, lazy mode and prompt length, confirm correct output, then run adjacent comparisons. Do not turn one machine into a hardware ranking or present an unmerged patch as a stable release. This remains a local candidate, not an imported benchmark: required operating-system and decode fields are missing.

Report: https://github.com/ggml-org/llama.cpp/issues/31943 Unmerged fix: https://github.com/ggml-org/llama.cpp/pull/31957