Adding logit penalty for "wait", "maybe" and "perhaps" to Qwen models improves their accuracy

Meta came out with a banger paper arxiv.org/pdf/2606.00206 , but it did not look at various quantizations supported in llama.cpp. So I did a run on 50 random MATH-500 questions ( huggingface.co/datasets/HuggingFaceH4/MATH-500 ) and ran it on various quantizations of huggingface.co/bartowski/Qwen_Qwen3.5-4B-GGUF and tried --logit-bias 466-2 --logit-bias 694-2 --logit-bias 1362-2 \ --logit-bias 1412-2 --logit-bias 1921-2 --logit-bias 1990-2 \ --logit-bias 2086-2 --logit-bias 2361-2 --logit-bias 2441-2 \ --logit-bias 2493-2 --logit-bias 2892-2 --logit-bias 3222-2 \ --logit-bias 3315-2 --logit-bias 3384-2 --logit-bias 3404-2 \ --logit-bias 3482-2 --logit-bias 3655-2 --logit-bias 4213-2 \ --logit-bias 4370-2 --logit-bias 4598-2 --logit-bias 4611-2 \ --logit-bias 4808-2 --logit-bias 5752-2 --logit-bias 6970-2 \ --logit-bias 7014-2 --logit-bias 7643-2 --logit-bias 8106-2 \ --logit-bias 10179-2 --logit-bias 10451-2 --logit-bias 11746-2 \ --logit-bias 13264-2 --logit-bias 13428-2 --logit-bias 14673-2 \ --logit-bias 15029-2 --logit-bias 16036-2 --logit-bias 21143-2 \ --logit-bias 21979-2 --logit-bias 33955-2 --logit-bias 35999-2 \ --logit-bias 36563-2 --logit-bias 37201-2 --logit-bias 37781-2 \ --logit-bias 41484-2 --logit-bias 62586-2 --logit-bias 66073-2 \ --logit-bias 73071-2 --logit-bias 84485-2 --logit-bias 85152-2 \ --logit-bias 95500-2 these correspond to the paper's overthinking markers: [ " perhaps", " maybe", " wait", " Wait", " actually", " hold", " Hmm", " hmm", " Alternatively", " alternatively", " However", " however", " instead", " Instead", " But", " but", " though", " although", " yet", " rather", " unless", " otherwise", " nonetheless", " nevertheless", " regardless", " still", " anyway", " Or", " or", " either", " whether", " uncertain", " unsure", " possibly", " might", " could", " another", " different", " reconsider", " rethink", " backtrack", " retry", " revisit", " doubt", " confused", " wrong", " mistake", " error", " incorrect" ] Here are the results, surprisingly even BF16 leads to better accuracy. Caveats being this is one test on one model. Try it out and see it helps! Format Accuracy: baseline → penalty Reasoning tokens BF16 74% → 84% −19.4% Q8_0 76% → 80% −11.0% Q4_K_M 60% → 66% −14.8% Q3_K_M 52% → 66% −17.5% Q2_K 12% → 24% −11.5%