Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME
I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3_S with just 12GB VRAM and 32GB system RAM, achieving 20-30 tok/sec decode (Q2 achieves 39-45 tok/s) & 300 to ~90 thousand tok/sec prefill @ 131k context, on a custom fork of Strata. This fork has tonnes of architectural changes, all are very experimental and will probably break. But the performance makes up for it. This feels like local Opus in some regards, on sub 2k in compute. github.com/bodhi37/Qwen3.8-Flash-12GBRAM-32GBVRAM-SSD-Recipe github.com/bodhi37/strata
评论
?
参与讨论