Qwen3.8-Flash-Next @ 1.66bpw-exl3 STABLE under 36gb
Fits entirely in a 5090, no offloaded experts. Super-solid sub-2bpw! ( Liquid cat demo video generated by the quantized model) From the Cat's mouth: Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization. huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3
评论
?
参与讨论