Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp

If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do with Qwen3.8-27B's native reasoning effort capability! Selecting any value for reasoning in the web-ui (default, (off), low, medium, high or max) just introduces different hard cap values and and will truncate your reasoning, if those values are reached. With the exception of the "off"-option which disable reasoning at all and "default" and "max" without any capping. The reasoning effort is independent from that and really changes the thoroughness and the analytic reasoning skills and therefore can massively influence the output quality of the model, instead of just capping reasoning tokens! In older llama.cpp versions, it needs to be set via: --chat-template-kwargs "{\"reasoning_effort\":\"medium\"}" \ quotation marks are escaped for Windows) For up to date versions, if one doesn't like the chat-template-kwargs variant, it can be set with: --reasoning-effort medium Or, for external apps, via "reasoning_effort": "medium" API request. Options are: low, medium and xhigh (default) for any of the above methods. Did only test the chat-template-kwargs variant though!

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论