slow streams, giant outputs, and dropped connections on DeepSeek V4-Pro

I run V4-Pro in prod through their Anthropic-compatible endpoint for an agentic coding product. A few things keep coming up. I reached out to support but I got no response. Output speed. I measure ~33-60 tok/s on V4-Pro. When the model decides to write a 12-16k token response, that's a single request running 6-8 minutes. Is this just what it is? Huge outputs that hit max_tokens mid tool call. Both Pro and especially V4-Flash sometimes go on very long monologues before the tool call. When that hits max_tokens, the tool call is lost and the model restarts from scratch on the next request. On one identical task, Flash produced 196k output tokens vs Pro's 43k and ended up slower AND more expensive than Pro. Raising max_tokens helps Flash but doesn't fix the underlying verbosity. Mid-stream TCP resets on long streams. Requests get a connection reset partway through long streaming responses. The Anthropic SDK retries 429s but not dropped streams, so I had to build custom logic. How often are others seeing this? Caching quirks: cache_control markers are ignored (auto-cache only), cache write tokens always report 0, and the prefix cache is byte-exact on the tools block with no canonicalization, so your tool JSON has to serialize identically on every request or the cache dies right after the system prompt. That last one cost me ~4.7x input tokens until I found it. Are these known issues with workarounds, or is everyone just living with them?

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论