An analysis of the inference non-convergence and infinite loop issues I encountered when using DeepSeek-V4-Flash-0731 via the Ollama Claude Pro plan (based solely on personal test data).

DeepSeek-V4-Flash-0731 on Ollama Cloud: Thinking-Loop Root Cause and a Tool-Calling Mitigation

Abstract

The infinite thinking loop observed when running deepseek-v4-flash:0731 on Ollama Cloud at high/max reasoning effort is caused by a serving-stack bug in the DSpark speculative-decoding path at draft depth 5 (dspark_block_size), compounded by Ollama's llama.cpp-based deployment (which lacks the correctly-configured vLLM+DSpark stack used by DeepSeek's official API). FP8 quantization is the standard deployment format and is NOT the root cause — the bug reproduces on full-precision weights. Injecting mandatory tool-calling requirements into the agent's instruction file (agent.md) mitigates the non-convergence at high/max effort by converting unbounded thinking into bounded think→act→verify cycles.

1. Symptom

• Reported upstream: GitHub ollama/ollama Issue #17892 — deepseek-v4-flash:0731 (cloud) repeats the same thinking block indefinitely on a complex agent task: the same reasoning paragraph was generated 221 times over ~1m45s, ending in failure with zero usable output and only 4 tool calls (2 failed).
• Reproduced in our testing (direct API calls to https://ollama.com/v1/chat/completions, model deepseek-v4-flash:0731): at reasoning_effort: "high" on a complex implementation task (thread-safe LRU cache with TTL), 5/5 requests produced zero content, with thinking lengths of 20,171 / 31,899 / 29,751 / 27,377 / 31,157 characters respectively. At reasoning_effort: "max" with a 65,536-token output budget, the model generated 286,546 characters of thinking and zero content (finish_reason: "length").

2. Root Cause Analysis

2.1 Primary cause: dspark_block_size = 5 serving-stack bug

The DeepSeek-V4-Flash-0731 checkpoint ships a DSpark speculative-decoding module (Multi-Token Prediction heads integrated into the architecture). Its native draft depth is strictly 5:

"The checkpoint dspark_block_size is strictly 5" — DeepWiki, DGX-Spark runbooks

Depth 5 is a broken serving-stack value, confirmed by a controlled depth sweep under identical bursty agentic load (streaming, thinking on, 20-tool schemas) in HuggingFace Discussion #39:

DeepSeek officially confirmed the fault is in the serving stack, not the model — HuggingFace Discussion #50:

"The depth-5 corruption reports were a serving-stack bug, not this model... The model is innocent. The fault was an application/serving-stack bug."

Independent reproduction: Anemll/dspark-vllm-gx10 Issue #3 — a long-context request in high-thinking mode generated a repetitive reasoning loop consuming the full 65,536-token budget, returning HTTP 200 with finish_reason="length", message.content=null, and 262,689 characters in the raw message.reasoning field.

The official DeepSeek-V4-Flash-0731 README recommends num_speculative_tokens: 7 — explicitly avoiding the broken native value 5.

2.2 Deployment difference: llama.cpp vs vLLM

• Ollama Cloud serves the model on llama.cpp. llama.cpp merged DSpark support only on 2026-08-02 (PR #25784), three days after the model's release — the integration is immature and, per the evidence above, runs at the broken depth 5.
• DeepSeek official API and relay-station (inferai) serve the model on vLLM with DSpark correctly configured (--speculative-config '{"method":"dspark","num_speculative_tokens":7,...}' per the official README). On these deployments, max effort converges reliably (verified: relay-station produced a 13,392-character complete design where Ollama Cloud produced zero content).

2.3 FP8 quantization: standard, NOT the root cause

• Ollama Cloud serves the model at FP8 (verified via GET https://ollama.com/api/show → quantization_level: "FP8", parameter_size: 304,180,418,494, context_length: 1,048,576).
• FP8 is the standard deployment format for this model: the official vLLM recipe uses --kv-cache-dtype fp8, and the reference GGUF builds are FP8.
• The bug reproduces on full precision: the HF Discussion #39 depth sweep ran on sglang / 4× RTX PRO 6000 (sm_120) with full-precision weights. Quantization therefore cannot be the cause of the thinking loop.

2.4 Trigger condition

The corruption is triggered by long thinking chains (complex open-ended tasks). Short thinking does not trigger it:

No API parameter bypasses the bug. Tested and ineffective: all sampling parameters (temperature 0.0–1.0, top_p 0.8–1.0, frequency/presence penalties), system-prompt directives, max_tokens from 3,000 to 65,536, native-API repeat_penalty, and speculative-decoding disable options (not exposed by the API).

3. Experimental Evidence (direct API, deepseek-v4-flash:0731, https://ollama.com/v1)

Key contrast (rows 1–5 vs 7): the identical complex task at high effort produces 0/5 convergence without tools (thinking 20K–32K chars) but converges with tools present (thinking collapses to 221 chars, content produced, tool call issued). The presence of callable tools converts unbounded thinking into a bounded think→act→verify cycle; each tool result anchors the model and breaks the loop.

4. Mitigation: Mandatory Tool-Calling in agent.md

Injecting structured, mandatory tool-calling requirements into the agent instruction file mitigates non-convergence at high/max effort. The following five rules were added to the global agent.md (AGENTS.md) and verified in a live agent session:

1. Every turn MUST end with a deliverable: a tool call, code, a file change, or a direct answer. Thinking alone is never a complete turn.
2. State the plan ONCE at the start; subsequent turns reference "the plan" and act. Never restate a plan already stated.
3. Thinking exceeding ~1,500 chars without a deliverable = looping. Stop immediately; produce a partial result or call a tool.
4. When stuck, run a tool (grep/read/test) to gather facts instead of re-analyzing in your head.
5. Break tasks into steps of ≤5 tool calls, each with a verifiable deliverable.

Mechanism: rules 1, 3, and 4 force the model to act (call tools) rather than think indefinitely; rule 2 eliminates plan-restatement waste; rule 5 bounds each step. This does not reduce thinking intensity — it converts thinking into action, which is the verified convergence mechanism (row 7).

Scope limitation: this mitigation is effective in agent contexts with tools (pi, DeepSeek Harness). It does not fix the raw API behavior (no tools → still explodes). The definitive fix is deployment-side: Ollama must move the DSpark draft depth off the broken value 5 (tracked in GitHub Issue #17892) or disable speculative decoding.

5. Conclusion

1. The thinking loop on Ollama Cloud is a serving-stack bug at DSpark draft depth 5 (dspark_block_size), officially confirmed by DeepSeek as "not this model."
2. FP8 quantization is standard and not the cause — the bug reproduces on full precision.
3. The llama.cpp deployment (vs DeepSeek's correctly-configured vLLM+DSpark) is the environment where the bug manifests.
4. Mandatory tool-calling in agent.md is an effective agent-side mitigation: it converts unbounded thinking into bounded think→act→verify cycles (verified: thinking 30K→221 chars).
5. The definitive fix requires Ollama to correct the speculative-decoding depth (Issue #17892); until then, max effort is only reliable on vLLM-based deployments (DeepSeek official, relay-station).

References

1. HuggingFace Discussion #39 — Reasoning loops (depth sweep)
2. HuggingFace Discussion #50 — Official: serving-stack bug, not the model
3. GitHub ollama/ollama Issue #17892 — thinking output loops indefinitely
4. GitHub Anemll/dspark-vllm-gx10 Issue #3 — repeats reasoning until max_tokens with DSpark MTP=5
5. DeepWiki — Speculative Decoding: MTP, DFlash, and DSpark (dspark_block_size strictly 5)
6. DeepSeek-V4-Flash-0731 official README (vLLM recipe, num_speculative_tokens: 7)
7. Ollama Cloud model details via /api/show (FP8, 304,180,418,494 params, 1,048,576 context) — verified directly
8. llama.cpp DSpark support merged 2026-08-02 (PR #25784)

submitted by /u/South_Can_3680