Deepseek 4.1 flash has context problems.

I think the big split in how people review 4.1 might come down to context size. I'm developing my own coding agent and because of that I pay quite a lot of attention to bugs and misbehaviors. Deepseek 4.1 flash goes stupid at around 125k tokens input context. The evidence I have are how it does tool calls. My agent uses the argument "cmd" for the shell tool. Deepseek 4.1 is trained mostly on Claude which uses the argument "command". So the model using the argument "command" means that it forgot the system prompt and is reaching for what it knows by instinct and not what the actual tool schema is. input tokens tool calls bad arguments % bad arguments 0–25k 1,319 3 0.2% 50–75k 3,803 81 2.1% 100–125k 2,083 73 3.5% 125–150k 1,770 138 7.8% 175–200k 872 116 13.3% 250–300k 668 196 29.3% >300k 580 294 50.7% In some sessions I've had almost 70% error rate after 500k token context, but too small sample size to include in this table. I've done other experiments with system prompts and can reliably make it start ignoring explicit system prompt instructions at bigger context sizes including prompt injection from reading files or web searches (this is a big problem). This is compounded by the model being very eager to verify things over and over again with tools which adds to the context. To make it behave you should treat it like a 125k context window model, not 1M and then it will behave quite well. But that in itself can have a big problem - the stupidly cheap cache reads is part of what makes this model great and constantly rewriting the context to keep it under 125k is very bad for cache behavior and will actually increase the cost rather than letting it run stupid for an extra hour (it will eventually figure things out).

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论