No, Engrams won't let you run 1T models locally. It does something even better.

Ever since Qwen 3.8 Flash Next dropped, there's a misconception going around that N-gram tables will let people run 1T+ parameter models on a single server with 980B parameters offloaded to SSD. I'm here to disappoint you: it won't. But what it will actually do for local models is even better. At its core, Engram is just an embedding table with a longer key. Instead of indexing a static vector by a single token ID, you index it by the last 2-3 tokens, an N-gram. "New York" gets its own memorized vector, "the United" gets its own, and so on. Hash the N-gram, fetch the vector, feed it into the network. O(1), constant time, no FLOPs. Why bother? Because a surprising amount of what a transformer does in its early layers is reconstructing static crap from scratch: how entity names are spelled, formulaic phrases, common collocations: "New" + "York" = Wall Street, delis, rats, subways. But every time the model needs to recall a multi-token entity, it burns several layers of attention and FFNs re-assembling something that is, frankly, a database lookup. Engram moves that job to an actual database lookup so the neural layers can spend their depth on actual reasoning. So instead of spending a bunch of layers "rederiving" the meaning of multi-token phrases like "New", "York", Engrams enable that lookup to be performed instantly. This is why Qwen 3.8 Next can carry 51B parameters of N-gram embeddings while only activating around 6B per token: the table is cheap to query, so you can make it enormous and have it live in RAM or SSD. Now the part nobody understands: the lookup is "dumb". The key is just the last 2-3 tokens. Your 200k tokens of context have zero influence on what gets retrieved. The wider context can accept or reject whatever vector the N-gram fetched, but it can't change what was fetched. Engrams are used to store "meaning", similar to embeddings. It doesn't replace reasoning or computation. When an Engram model sees "import std", it doesn't suddenly gain years of C++ programming experience from the Engram vectors. The table memorizes, the transformer reasons. And you can't fix this by cranking N up either. The higher the N, the rarer that specific N-gram is in training data, so each entry gets less and less training signal. The paper's own ablation found that allocating capacity to 4-grams "dilutes capacity from the more frequent 2/3-gram patterns", so you can't scale the Engram embeddings up to 500B without it literally becoming a waste of space. But here's the better news: Engrams are an incredible architectural innovation. The fact that Engrams allow models to offload multi-token "meaning" derivation away from their active parameters means that smaller models will become much smarter; this is why I think this is one of the best architectural developments for local models in years. A 27B model has always had to spend its parameter budget on performing two jobs at once: actually reasoning, and memorizing static patterns that a lookup table could hold. That's a big part of why smaller 4B or 7B models feel dumb even on tasks well within their reasoning ability. Engram splits those jobs: the knowledge moves into a table that costs nothing to query and every active parameter gets freed for reasoning. That's the big innovation that everyone should be excited about: Smaller models that will as intelligent as Opus or Sol today, not bigger ones.