Qwen3.8 Flash Next Unsloth quants KLD&Top1% with default wiki, English books and Finnish books.

I was curious how quantization affects Finnish language ability and why Q38FN’s Q8 quantization got such poor accuracy numbers. So I set the machine running in the background. The Finnish corpus is three books from Project Lönnrot, and the English one is the same books from Project Gutenberg. Wiki is wiki. The first observation is that Finnish really does take a hit from quantization. If KLD is said to become noticeable after 0.02, then even at Q8 we’re far from that. This measurement doesn’t say anything about how well the model knows Finnish in its bf16 form, only how much worse it gets from there. If a rarer language hasn’t been drilled into the model with large weights, it starts to show when bits are dropped from the precision. The second observation is that some of the people who quantize models use their own (non-public) corpus to evaluate accuracy. They justify this by saying that wiki has already been trained so well into the n-gram that it would otherwise come through too much. The results are somewhat mixed. KLD on the wiki data is better, but top-1% is worse. In fact, the Q8 on the English data is starting to be what I expected from that level of quantization. I run a hybrid (“chimera”) model where I’ve swapped Unsloth’s Q4 for a Q8-level n-gram. Someone had measured that it wouldn’t be worthwhile, but by my measurements there’s a clear benefit. The hybrid model lands midway between the Q4 and Q5 models, which is a good result considering I run the n-gram from disk, i.e. it doesn’t load memory at all. A clear observation is also that the size of the n-gram has no effect on Finnish ability. The charts also show that even a Q5 model would be very close to the Q8 model. To some extent I miss Gemma-4 31B’s perfect Finnish, but on the other hand Q38FN’s coding skills are in a whole different class.