I pushed Qwen3.8-27B to 2.000 prefill per second and 132 decode per second on A RTX 3090.

Yoyo I'm back with updates to the fastest inference engine with minimal quality loss for Qwen3.8-27B. The last few weeks I've been optimizing decode speed and I don't think it can be pushed further, until a newer/better drafter is invented. So I focused on prefill, which I this morning was around 1.300 per second at 4k and now is just below 2.000. The main improvement came from a custom kernel, which matches the quality of fp32 with 0.99997 similarity at int8. Try all of the improvements here: github.com/syv-ai/qwen38-27b-rtx3090