Keeping vLLM's Prefix Cache Warm Between Agent Turns

From 55% to 95% cached

I’ve been playing with a few different ways to host Qwen3.8 locally. I’m aiming for something that can replace Claude Code for most of my tasks. Right now the 27B runs on two RTX 3090s under vLLM, and it’s usable, but the first day was rough. In the morning the wait before the first word averaged about half a minute, and some replies took several minutes. The time went into rereading large parts of every prompt from scratch, because the cached copy from the previous turn had been thrown away or no longer matched. By the evening the server was keeping almost everything from one turn to the next.