Context management in long agent threads
Pardon the long preamble, hopefully you will read it and respond because I am very curious about people's thoughts on this. If you've been at this for the last few years, does this observation resonate at all? I'm using Qwen 3.8 27b with my GuideAnts stack to do long running coding tasks baking off against Grok 4.6 in Cursor. After about a week and a half, 27b on extra high is, hands down, the winner. Grok 4.6 spits out tokens faster but the quality of 27b wins and I get a better final result in less time. One thing I know is that how a given harness uses a model matters and its hard to reach conclusions. What I should do is use Grok 4.6 from GuideAnts, but just reflecting on what I can see with my own eyes is that Cursor loves to summarize the conversation well short of the maximum context size they show in the UI. What I do in my harness is evict old tool calls to relieve pressure followed by thinking and then, if it is at max size with just user and assistant messages, I consider it full and start a new task or thread. Sometimes, I will summarize the previous conversation first, but I decide that and also what the summary should focus on, and there is no auto-magic compression ever. It could be that the reason Grok is sucking in Cursor is because I am testing long tasks and it stomps the context in a way that is too lossy. What I find is that I can keep a thread going at 272k for a very long time because it can always choose to retrieve data by doing new tool calls if the call itself was evicted previously and it needs the info, and it can always 'think again' if the overall context of the messages is intact. Things like compaction (and RAG) which we used to do to prepare the input and manage 16k-64k context windows over long threads seem to me like something people should mostly stop doing. TLDR; Maybe compacting context with conversation summaries is a bad idea now. If you've been at this for the last few years, does this observation resonate at all?