Qwen3.8-Flash-Next MTP merged in ik_llama.cpp (integrated head or separate -md file)... 45 → 90 tok/s on a 5090 + 128GB, works down to a 12GB 4070

Been building on Flash-Next since release week and this landed yesterday: ik_llama.cpp merged qwen4exp MTP support (PR #2369, mine, reviewed and tested by four other people on their own hardware). It's on main now, no fork or patch needed. Figured I'd post since the last couple threads had people saying MTP for this model only exists as an unsloth fork PR... there's another path. Flash-Next ships a 2.6B MTP head that the public converters were dropping. With it loaded the model drafts its own next tokens from its hidden state and then verifies them, so output is identical to running without it (greedy is exact-match against the target's own argmax). On code I get 93-99% draft acceptance per position, prose is more like 60-65%. Numbers so far, decode tok/s, no MTP → MTP. My box, RTX 5090 + 128GB DDR5 with the experts on CPU (-ncmoe 38): 45 → 90 on coding traffic steady state, with ngram-mod chained in front. treo on an RTX Pro 6000: 85 → 113 on code and 82 → 109 on extract, but story went 83 → 59 at 65% acceptance, so it is not a free win on prose yet. joelfarthing on a 12GB 4070: 9.5 → 12.5 on code at n_max=1, break-even on story. A 3090 replicated it in the PR thread, and a 5060 Ti running a fine-tuned target against the base head went net negative... pairing matters. Caveats: opt-in, single slot for now (-np 1), and --jinja drops acceptance because the template turns thinking on at xhigh by default and reasoning text drafts like prose. great for agentic coding, meh for creative writing on a small card. Build is a stock CUDA build: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 cmake --build build --config Release -j What I run (integrated head file, the head is blk.48 inside the GGUF, no second model): llama-server -m Qwen3.8-Flash-Next-MXFP4-ngramQ8-NextN.gguf -ngl 999 -ncmoe 38 -fa 1 -c 196608 -ub 512 -ctk q8_0 -ctv q8_0 -np 1 -t 24 -tb 32 --jinja --spec-type ngram-mod:n_min=4 --spec-type mtp:n_max=4 --spec-ckpt-mode gpu-fallback -rtr -muge If you already have an unsloth or other quant downloaded you don't need to re-pull 100+ GB, the separate head route works on the same code: -md .gguf --spec-type mtp:n_max=4 . dzannotti's head was tested by two of the reviewers and joelfarthing's 4070 numbers are with his own IQ4_KT head (his repo has ik_llama-native KT quants of the target too). Haven't tried unsloth's "shared" MTP shards on this path yet, those reuse the token embedding so it's a different layout. PR with all the receipts: github.com/ikawrakow/ik_llama.cpp/pull/2369 My integrated-head MXFP4 files: huggingface.co/jamesrogers/Qwen3.8-Flash-Next-MTP-MXFP4-GGUF ji-farthing's ik_llama KT quants + head: huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF dzannotti's head: huggingface.co/dzannotti/Qwen3.8-Flash-Next-MTP-GGUF Curious what you measure, especially anything AMD!!