We beat Mem0, Zep and Letta on two memory benchmarks. The score isn't the interesting part

I've been building a memory/context layer called BrainAPI for a while now, and we just landed on top of the two benchmarks we've run so far. I want to talk about it, but honestly the numbers are the least interesting thing here. The part I keep thinking about is how fast it happened , and what that says about where the actual bottleneck in this field is. First, the boring facts so nobody thinks I'm hiding the ball: LoCoMo : BrainAPI 95.39%, Mem0 92.5%, Zep 80.32%, Letta 74% BEAM1M : BrainAPI 78.97%, Mem0 64.1%. Zep and Letta haven't published here. That's it. Two benchmarks. I'm not going to pretend that's a complete picture. LoCoMo is fairly saturated at this point and it leans on an LLM judge, so a couple of points at the top is not the same as a couple of points in the middle. BEAM1M is the one I actually care about because it stresses the long horizon. I'm currently working toward BEAM50M and LongMemEval , and I'll post those whether they look good or not. Runs and reports are in the repo if you want to poke at the harness: github.com/Lumen-Labs/brainapi2 (the benchmarks folder), summary here: research.brain-api.dev The thing I actually want to talk about Two years ago, doing this kind of work looked like: go find the relevant papers. Which is already a project. You burn days just figuring out which twelve of the four hundred results are the ones that matter. Then you read them. Then you sit there trying to translate "we propose a temporally-aware episodic buffer" into something that fits into the retrieval path you already have, half of which doesn't apply and you only find out after you've built it. That loop was months. Not because the ideas were hard, but because the search and translation around the ideas was slow and lonely. Now: Cursor wired into an arXiv MCP, a set of skills that encode how I want the reasoning and the workflow to actually go, and a lot of leaning on plan mode before anything gets written. The paper discovery stops being a bottleneck. The "how does this apply to my architecture" step, which used to be the expensive one, becomes a conversation where the thing already has my codebase in context. Weeks, not months. Some pieces, days. And here's what I take from that. The model wasn't the constraint. Nobody handed me a smarter model between "this takes months" and "this takes weeks." What changed was the harness: retrieval into the right sources, structured context, workflows that reason in a shape I chose, planning before execution. Same model, radically different output. I think this generalizes, and I think it's the most under-discussed thing in the space right now. Every time an agent fails in production, the reflex is "wait for the next model." But go look at the actual failure. It forgot something from twelve turns ago. It couldn't connect two facts that live in different documents. It confidently answered from a chunk that was semantically close and factually wrong. None of those are intelligence problems. They're infrastructure problems. That's the bet BrainAPI is making, and why I built it as an event-centric graph rather than another vector store. When you keep who did what, to whom, when , instead of flattening everything into "A is related to B," multi-hop questions become answerable and the answer arrives with the path that produced it. You can inspect the walk instead of trusting a nearest neighbor. That's the context piece of the infra. Somebody's going to build the other pieces. What I'm curious about For those of you running agents in production: when it breaks, is it actually the model, or is it the plumbing? Be honest. Which memory benchmark do you personally trust? I have my doubts about all of them and I'd rather hear yours before I optimize toward the wrong one. Anyone else moved their research loop to MCP-connected tooling? Did you get the same compression, or am I just describing my own previously-bad process? Happy to go deep on the harness, the graph design, or the benchmark methodology in the comments. Roast the numbers if you want, that's kind of why I'm posting.