← News
Runtime

llama.cpp b11985 brings Metal decode changes, not a universal speedup

Metal changes merged October 29 are included in b11985, covering MXFP4 mat-vec, elementwise operations and GLU. Earlier PR tests are not new independent reproductions; upgrades still need controlled A/B testing.

Source: llama.cpp release and contributor PR

On October 29, 2026, llama.cpp merged Metal PR #31075. Build b11985, published at 17:10 UTC that day, includes the changes. The new event is upstream integration, not a newly measured performance result. The PR was opened on October 24; merging it does not turn its earlier benchmark table into measurements from the latest 24-hour window.

The first change targets MXFP4 matrix-vector operations during single-token decoding. FP4 values are decoded with an fp16 bit trick instead of a threadgroup lookup table, while packed nibbles are loaded as aligned 32-bit words. The release notes say the kernel no longer uses threadgroup memory. The PR also identifies an MoE expert path using the same function. This is a format- and operator-specific implementation change, not a blanket improvement for every quantization.

Contiguous unary and row-broadcast binary operations now use up to 256 threads per threadgroup rather than one. GLU rows are split across multiple threadgroups, so a single-row decode no longer assigns the entire operation to one group. Benefits depend on the operators a checkpoint actually exercises; a Metal update alone says nothing conclusive about every Mac.

The original PR contains interleaved M3 Ultra comparisons and backend tests. Those are earlier measurements supplied by the contributor, not independent AraoAI reproductions or coverage of every Apple chip. This update verifies merge status, release identity and implementation details. We are not importing the old results as fresh measurements or turning them into a hardware ranking.

Keep the previous binary, checkpoint hashes and launch arguments. Measure prompt processing and decoding separately, and inspect outputs. Hold weight quantization, KV precision, context, batch size and warmup procedure constant. A subjective chat impression cannot isolate caching from a kernel change. A new build is a reason to test, not a reason to replace a working setup without comparison.

Sources: b11985 release; original PR #31075.