Qwen3.8-Flash-Next Q2_0 in Strata on a Radeon 780M (gfx1103): production config and measured numbers

Qwen3.8-Flash-Next Q2_0 in Strata on a Radeon 780M (gfx1103) — production config and measured numbers

Measured 2026-11-04 on a running production instance. Everything below is from that machine, not from a model card.

Hardware

• Ryzen 7 255 (8C/16T), Radeon 780M — gfx1103, RDNA3, 12 CU
• 64 GB DDR5-5600 (2x32 GB SODIMM)
• UMA carve-out 2 GB, GPU cap set with ttm.pages_limit = 48 GiB (amdgpu.gttsize is deprecated)
• Ubuntu, kernel 7.0.0-42, amdgpu firmware from ~Oct 2026

Software

• Strata 0.2.9, HIP backend built for gfx1103
• ROCm 10.1.0 SDK from a local tree (HIP runtime 7.16.26385) via LD_LIBRARY_PATH; the system ROCm 7.1 is not used
• hipBLASLt tuning table for gfx1103 (143 rows, calibrated on this chip)

Model

Qwen3.8-Flash-Next GSQ-RCO Q2_0 (ISTA-DASLab), native pack with the experts read in place, PLE table from the second shard, MTP draft layer built with the project's mtp_fetch.py / mtp_pack.py / mtp_rt.py.

Serving configuration (exact)

--pack packs/q2_0
--native .../Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf
--ple-gguf .../Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf
--expert-profile data/expert-profile.bin
--expert-cache 8500
--prefill auto
--spec 2
--mtp mtp/rt
--max-context 65536
--kv int8
--kv-resident 65536
--vram-reserve-mib 700
--pool-workers 2
--control-vector cvec-v1.gguf --cvec-mode project

Environment (systemd drop-in, bit-changing opt-ins):

STRATA_HIP_WMMA=1
STRATA_PF_GEMM=1
STRATA_PA_FAST=1
STRATA_SELECT_WMMA=1
STRATA_PF_FUSED=1
STRATA_HC_UPMIX=1
STRATA_HIPBLASLT_TUNING=
LD_LIBRARY_PATH=/lib

Measured throughput

Fresh prompts with a unique nonce at the start (so nothing is served from the prefix cache), greedy, 256 max tokens. Prefill and decode are the server's own timings fields.

Memory at this config: RAM 58 of 59 GB used, expert-cache hit rate 89%, GTT inside the 48 GiB cap. Context 65,536 with the whole KV resident.

What actually moved the numbers

1. The WMMA opt-ins as environment variables — +24% prefill at 4K, decode neutral, zero GPU resets over long runs. They are opt-in and bit-changing; measure them on your chip before trusting them.
2. Expert cache sized in steps of 500, judged on MemAvailable read AFTER the filling the GPU's expert cache line. 8,500 slots = 10.94 GiB. 9,000 slots was slower (17.9 tok/s) and left under 1 GB free. 11,500 slots caused a global OOM and a kernel panic (amdgpu_hmm_range_valid) — 5 minutes of downtime. Reading free memory while the cache is still filling gives a number that is several GiB too optimistic.
3. --spec 2 beats --spec 4 on this chip (15.6 vs 14.2 tok/s in the interleaved test), and --spec-min-p 0.5 costs ~4.5% here — the engine default is better on gfx1103.
4. --kv-resident 65536 keeps the entire KV in GTT, which is why decode holds ~21 tok/s at 65k context instead of collapsing to ~12 as it does when the KV spills.
5. It was the firmware, not the runtime. Every ROCm crash we had (MES REMOVE_QUEUE -> GPU reset, CUBLAS_STATUS_INTERNAL_ERROR, MUL_MAT_ID failures) came from an amdgpu firmware blob that predated the GC 11.0.1/11.0.4 MEC/MES update. Updating linux-firmware ended them; no runtime swap was needed.

Pitfalls worth knowing before you copy this

• Removing --vision from the args is not enough. If the config still contains a vision block, the server logs the config has a "vision" section but no --vision in its args: added --vision and loads the encoder anyway (~1.3 GB GTT). Remove the block; verify with /v1/models -> input_modalities: ["text"].
• The expert arena is MAP_HUGETLB, all-or-nothing. vm.nr_hugepages must be ceil(experts_bytes / 2MiB) for the exact pack; otherwise the engine silently falls back to the file cache and can take the kernel down under memory pressure. Check the expert arena: line at startup.
• Readiness is not "the port is open". A restart takes ~2.5-3 minutes (31.6 GiB of experts at ~0.84 GiB/s) and /v1/models says loaded before the expert cache is filled. Wait for the filling the GPU's expert cache line plus one real request.
• Never start two large llama.cpp/Strata services at the same time on this class of box — instant OOM.
• If you serve this to an agent runtime, mind the context floor. Hermes (our client) refuses any model under 64k context outright, and its compaction threshold is a fraction of the window (0.38 x 65,536 = 24,904 tokens), so the two numbers move together.

For reference: same chip, same model, other engines

• llama.cpp Vulkan + MTP on a comparable 780M box: ~160 tok/s prefill, 7-8 tok/s decode.
• Strata community report on gfx1103 at 128K context: 277 tok/s at 4K, 13-14 tok/s decode.
• This config: 370-388 tok/s prefill, ~21 tok/s decode at 65k context.

The gap to the 128K report is mostly the context size and the expert cache, not a different engine build.