llama.cpp merges Qwen3.8-Flash-Next MTP support

llama.cpp merged PR #29761 adding multi-token prediction (MTP) speculative-decoding support for the Qwen3.8-Flash-Next checkpoint, with a reported ~55% decode-throughput gain on DGX Spark.

Why it matters

What you could build with it

Does it hold up?

Evidence-backed but setup-dependent: the PR author measured 28.36 to 43.88 tok/s (+55%) on DGX Spark with an IQ4_XS quant; community numbers (24.2 to 29.3 t/s on a Radeon 8060S, 43.8 to 73.5 t/s on an M5 Max) confirm real gains, while follow-up issues (MTP draft GGUFs failing to load on build b11058) show the path is still maturing.

Built with llama.cpp merges Qwen3.8-Flash-Next MTP support

• ik_llama.cpp PR #2369: qwen4exp MTP (NextN) self-speculative decoding supportgithub · Community fork PR enabling MTP self-speculative decoding for the qwen4exp architecture, building on earlier MTP work.
• How to Double Qwen 3.8 27B Text Generation Speedgeeky-gadgets.com · Third-party writeup showing MTP in llama.cpp lifting Qwen 3.8 from 7.9 to 17.1 tokens/sec on compatible hardware.
• Running Qwen3.8-Flash-Next 125B on Three RTX 3090s at 80 Tokens per Secondn1n.ai · Builder post on custom CUDA/C++ patches to llama.cpp reaching 80 tok/s for the 125B checkpoint on three RTX 3090s.
• ikawrakow/ik_llama.cppgithub · Community llama.cpp fork with qwen4exp MTP (NextN) self-speculative decoding support.
• ucicelos/flashnext-hybridgithub · 2 ★ · Build combining upstream llama.cpp qwen4exp support with an MTP draft-head port and tuned kernels.
• SergiioB/intel-arc-pro-b70-inference-cookbookgithub · Inference cookbook with a Qwen3.8-Flash-Next llama.cpp recipe reporting +42% decode from MTP.

Learn more

merged pull request →

First spotted on github: source.

More AI models