older Nvidia Maxwell running Strata

2 Quadro M6000 24GB (48GB VRAM) running Qwen3.8-Flash-Next-GSQ-RCO-IQ1 on Strata since I only had 32GB of RAM. Pretty impressive prefill numbers on this inference engine. The decode has always been decent for the maxwell flagships. This has 3.5 bits per weight so it is actually way more slower to run than a Qwen3.8-Flash-Next-GSQ-RCO-Q2. So if you have 40GB of ram, the Q2 will even be a faster model to run. I simply lower the restriction on the strata code to allow lower than architecture => 52 (rather than =>60). it is still running the same pipeline as the pascal gen cards. VRAM Hit rate is virtually 100% with all experts loaded on vram.

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论