Infernix - New Release!

Over the past few weeks, I've been working on various improvements to my inference engine Infernix - specifically optimised for running the latest Qwen architecture models on an RTX5090 on Windows and Linux. Windows is a first-class supported use case - you can just download everything you need to run on windows here: github.com/Wallawalla47/Infernix/releases/tag/windows-2026.11.01 Key improvements: New models: the new NVIDIA QAD FP4 version of Qwen3.8-Flash-Next (which operates faster than the NVFP4 model and can be switched to use a4 activations for even faster performance with --a4-activations): huggingface.co/wallawalla47/Qwen3.8-Flash-Next-NVIDIA-FP4-Local-Infernix the same GSQ-RCO-IQ3_S model supported by Strata (which offers some faster speeds than NVFP4 but I find at some quality cost): huggingface.co/wallawalla47/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-Infernix Swift 1.5 Qwen3.8-Flash-Next NVFP4: huggingface.co/wallawalla47/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-Dense8-Infernix Qwen3.8-Flash-Next Full FP8 (fairly slow, but offers the best quality being bit-exact to the official FP8 model from Qwen): huggingface.co/wallawalla47/Qwen3.8-Flash-Next-FP8-Infernix the new Ornith 1.5 122B NVFP4 model, which has impressive benchmark performance, but I'm not sure I'm sold on it yet: huggingface.co/wallawalla47/Ornith-1.5-122B-NVFP4-Infernix (and the uncensored version as well: huggingface.co/wallawalla47/Ornith-1.5-122B-UNCENSORED-NVFP4-Infernix ) DFlash2 drafting for Qwen3.8-Flash-Next models (use --spec dflash2 --draft-tokens auto --draft-model [Draft Model File Path - e.g. D:\Infernix\Qwen3.8-Flash-Next-DFlash2-Infernix.infernix]): huggingface.co/wallawalla47/Qwen3.8-Flash-Next-DFlash2-Infernix Added a new WebUI Launcher (beta), which provides an interface to download and configure models and a model switching API, so it functions similar to launchers like LMStudio or Unsloth and it also has a statistics dashboard, logging and a quick test chat interface (still early stages, so interface is quite busy and needs some refining, but is probably the easiest way to configure and use Infernix): on Windows just launch infernix-launcher.exe Added new auto-configuration of speculative decoding which uses calibrated settings to give the best decode tok/sec performance across a range of different types of text - simply use: --spec auto --draft-tokens auto --draft-model [Draft Model File Path] Lots of speed and quality optimisations - generally significantly outperforms Strata and occasionally outperforms Basalt, though it does avoid some of the quality sacrificing optimisations often used by default in Basalt and other engines like expert tail skipping or running the model at a lower quantisation than recommended for the model. You can enable some of these options if you like, such as expert tail skipping with --expert-tail-skip [Int] (7 is sometimes the default in other engines) or using 8-bit code paths (rather than the default 16-bit/4-bit recommended for the model) with: --a8-activations Adding a new --lossless mode that is always perfectly reproducible and uses bit-exact in-layer calculations to the precision the model was trained on (usually BF16) with FP32 logits. This is helpful as a reference for quality testing and I'm working on some more robust quality gating to ensure all optimisations and code paths do not impact quality in the same way as other engines often do. Some key performance figures: Infernix vs First token Decode Replay: mean first token Replay: wall time Qwen3.8-Flash-Next, NVIDIA's NVFP4 weights Strata v0.2.8 1.4-1.6× sooner +10-15 % to 128K, −4 % at 250K 3.1 s vs 12.3 s 5.7 vs 12.9 min strata-nvfp4 0.2.0-beta-nvfp4.3 1.3× sooner level to +3 % 3.1 s vs 10.8 s 5.7 vs 11.7 min Basalt c08284cb 5-8× sooner Basalt 7-23 % faster 3.1 s vs 7.2 s 5.7 vs 8.9 min Qwen3.8-Flash-Next, ISTA-DASLab's IQ3_S GGUF Strata v0.2.8 15 % sooner at 8K, level from 128K +3-12 % 3.1 s vs 6.9 s 5.5 vs 8.1 min Basalt c08284cb (Basalt is faster on this model but uses some lower precision math) Basalt 4 % sooner at 8K, 1.3× from 128K Basalt 1.4-1.7× faster 3.1 s vs 3.7 s Basalt sooner: 4.4 vs 5.5 min Qwen3.8-27B, the same NVFP4 artifact in both NInfer 1882c6adb 9 % sooner at 8K, 1.6-1.8× from 128K +11-17 % (own drafters), +2-3 % without 1.9 s vs 2.5 s 12.7 vs 16.4 min On the NVFP4 model, Basalt runs NVIDIA's BF16 dense layers at 6 bits (+10 % KL divergence by its own measure) and every activation outside the experts at 8 bits ; Infernix keeps 16. Infernix runs activations at 4-bit as recommended by NVIDIA for this model, but with the expert activations at 8 bits as Basalt runs them in decode ( --a8-activations ), Infernix decodes 199 / 185 / 268 tok/s against Basalt's 202 / 173 / 216 at ~8K / ~128K / ~250K and 175 against 166 at one request in the replay, and still reaches the first token 4-6× sooner. Strata's default route tail skip (it skips some missed experts) was off in all runs: it changes which experts run. On the IQ3_S model, all three hold the GGUF's weights as published, but Strata and Basalt compute every activation at 8 bits , and Basalt's decode reads the BF16 hyper-connection matrices from an 8-bit copy and takes the logits outside the top 64 from a 2-bit copy of the output head; Infernix keeps 16-bit activations, the BF16 matrices and the GGUF's head. At 8 bits ( --a8-activations ) Infernix reaches the first token as soon as Basalt, though Basalt is still a bit faster. Welcome any feedback via Issues and PRs on GitHub. I'll also look to add support for the new Qwen 4 models ASAP when they drop - hopefully this week!