Anyone managed to get Qwen 3.8 27B running smoothly on vLLM? Can't get rid of endless thinking
Title pretty much says it all. I’ve deployed Qwen 3.8 27B using vLLM on an RTX 6000 Pro (tried multiple vLLM releases and launch recipes), but I can't get it into a usable state because of crazy long reasoning passes. Regardless of the thinking effort setting (xhigh, medium, or low), it takes way too long to respond: xhigh: Spends up to 5 minutes overthinking basic questions before outputting anything. medium: Takes around 2 to 4 minutes. low: Still stuck thinking for 1–2 minutes. For comparison, running Qwen 3.6 or DeepSeek V4 Flash on the exact same hardware answers these identical prompts within 20–30 seconds. What I've tested so far: Quantizations: Tested both FP8 and NVFP4. Proxy / Effort Mapping: I'm running a proxy in front to map thinking efforts. I've validated this thoroughly across many other reasoning models, so I’m confident the mapping logic itself isn't the issue. vLLM Flags: Tried tweaking context size, MTP speculative decoding, and parser configs (recipe below). I've seen similar reports on the Hugging Face repo discussions and scattered Reddit comments, but at the same time, I see a ton of posts raving about how fast and great this model is. Right now, I just can't reproduce those good results. My latest startup recipe: --tensor-parallel-size 1 --max-model-len 262144 --kv-cache-dtype bfloat16 --gpu-memory-utilization 0.90 --max-num-seqs 32 --max-num-batched-tokens 8192 --max-cudagraph-capture-size 32 --enable-chunked-prefill --enable-prefix-caching --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --limit-mm-per-prompt '{"image":4,"video":0}' --mm-processor-kwargs '{"size":{"longest_edge":1048576,"shortest_edge":65536}}' --mm-processor-cache-type shm --default-chat-template-kwargs '{"preserve_thinking":false}' --speculative-config '{"method":"mtp","num_speculative_tokens":3}' --enable-prompt-tokens-details Has anyone encountered this endless reasoning behavior with Qwen 3.8 on vLLM? Is there a specific chat template tweak, system prompt hack, or generation param (like strict thinking_budget or token limits) required to stop it from burning through cycles? Appreciate any insights!