Running Qwen3 30B A3B at 50 tok/s on RTX 5060 Ti

Experimented with some custom CUDA and C++ code that can now run a Qwen3-30B-A3B at 50-54 tok/s at float 8 on an RTX 5060 Ti with only 16 GB of VRAM. This speed is roughly 50% improvement to llama.cpp which runs at around 33-34 tok/s (with n-cpu-moe). These speedups come mostly from combining SOTA solutions I saw in papers in NeurIPS, ICML, and EuroSys Engines like these allow for new local inference oppurtunities on consumer hardware, offering more private, cheaper, and greener alternative to centralized d