R9V Update: Created and adopted KVA projections based on Deepseek V4.1 Flash + HySparse2/MiMo-V3 for Qwen3.8 Flash Next. This is a game changer for models that don't natively implement it. 1.45-1.8...

Here's my first implementation of KVA projectors on QFN (just the uncensored model for now) the highlights are basically as follows for using the projectors at each different layer: Starting at layer 12, prompt processing speeds up 1.85x [1700 t/s -> 3150 t/s] at the tradeoff of increasing perplexity a total of +8% At layer 16, prompt processing speeds up 1.7x [1700 t/s -> 2900 t/s] at the tradeoff of increasing perplexity a total of +5% At layer 24, prompt processing speeds up 1.45x [1700 t/s -> 2500 t/s] at a tradeoff of increasing perplexity a total of +2.6% This method is different than the other KVA projectors I have seen for the following reasons: My method uses one full map per layer that uses Tikhonov regularization/ridge regression vs a per layer + training correction heads thats applied to 4 streams, then averaged out. This translates to higher accuracy and less perplexity, at the cost of more VRAM. The other methods use apx 400mb while mine uses 1.5ish GB. Other methods predict later layers keys, values, and inputs directly, while my method predicts strictly the inputs to the later layers, and depends upon the models actual weights to compute keys/values. Finally, the other methods I've seen are not variable by which layer implementation starts at (usually locked to 24 i believe), while my method is variable and allows you to determine your own risk tolerance for increasing PP speeds at the cost of increased perplexity. I have a lot of faith that this idea can be expanded and become hugely useful based off my initial indications. In less technical terms, its sort of like MTP for pp instead of tg, except it's not lossless. The error does get ingested by the model. Models like DSV4.1 and likely MiMo v3 are likely trained alongside this type of implementation, so they may be more tolerant to the ppl increase already. Models that havent been trained against this, like QFN, will continue to see that ppl increase where error occurs. Here's a summary of the BetterBench results. Metric Result Detail Prefill 3,600 t/s @ 64k tok Decode 74.7 t/s Weighted combined Concurrent 70.3 t/s @ 8 streams (48/48 ok) TTFT (P50) 338 ms Single stream Update (P99) 51.5 ms Stream stutter BUT WAIT, THERE'S MORE! Here's my 2nd implementation. Based off of the HySparse2 paper, it appears that they are using a similar method but multi-layered instead of single layer. Based off of this, I've built an initial early version of this. Here's what the preliminary results show: Multi-layer - 1.55x speedup at only a +2% of perplexity Using this, I strongly believe that this can be implemented for a total of 1.5x speedup while <1% ppl increase. If anyone wants to adapt this to other engines and models, just note that I found more training to be virtually worthless, it's purely architectural levers that need to move IMO. Currently this is in very early testing- R9V is updating with this capability and this projector is getting uploaded to HF, but it is NOT CONFIRMED STABLE. The V1 iteration of KVA Projectors is however stable. V2 projectors are behind a config flag ( --ced quality) that you can choose if you wish. Here is the HF repo for the full IQ4_XS QFN Model + MTP + KVA Projector (V1) - this one is directly usable in R9V now. huggingface.co/Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS Here is the repo with just the projectors, V1+V2, with a short explanation on how to get started on implementing this method in other VLLM projects huggingface.co/Dyluhn/Qwen3.8-Flash-Next-Uncensored-CED-Projector/tree/main NOTE- R9V is specifically built for 2x R9700 setups with significant host RAM. RAM usage floats around 50ish GB during use for expert storage. You'll likely need an SSD to handle PLE/en-grams at usable speeds, or just keep them in RAM. Enjoy! Join discord.gg/launch80 if you are interested in working on and with some of the latest and greatest implementations for RDNA4 (there's even better projects than this one in there) Also, need to acknowledge that huggingface.co/kishida was the first one, as far as I can tell, to determine that this feature can be borrowed from DSV4.1 separately from the model architecture. Bravo.