My system prompt is 100k tokens. What's the best way to compress markdown files for Web UIs?
TL;DR: I only use Web UIs (DeepSeek/ChatGPT/AI Studio). My system prompt .md file is 100k tokens. What's the best way to compress/optimize this to save context space without losing critical details? --- Hoping to get some advice on a workflow bottleneck. I’m currently hitting a wall with prompt limits and looking for some optimization strategies. My setup: I have a massive system prompt stored in a .md file. It contains all my instructions, reference data, rules, and background context. I use Web UIs exclusively (DeepSeek, Claude, etc.). No API calls, no local scripts. The issue: This single markdown file sits at around 100,000 tokens . Loading it into the Web UI eats up a massive chunk of the context window right off the bat[ 1 ]. Naturally, this leads to slower response times, the model forgetting instructions faster, and hitting usage caps way too quickly. I need to keep the core rules and data intact, but I seriously need to shrink the token count. What are the best practices or tools to handle this? Semantic compression: Are there reliable prompt-compressors or techniques to condense data without losing structural instructions? Formatting tweaks: Does switching from Markdown to JSON, XML, or pseudo-code actually save a meaningful amount of tokens? Web UI workarounds: Do native features like Claude Projects or Custom GPTs handle large files better in the background, or do they still front-load the entire token weight into the chat history? Would love to hear how you tackle token optimization for heavy workloads on web interfaces. Thanks in advance for any tips!