RAG, Knowledge Packs and Your Notes

With LLMs, for better or worse, the cat is out of the bag. There is no going back. What is still up for grabs is how we use them.

At Ente, we believe in a future where LLMs run entirely on your device. Useful for everyday tasks, with privacy and control built in.

A bunch of companies are building models for that future. With Ensu, we are working on the software around those models to help them do more.

Ensu Packs

Small models can do a lot. But the ones that fit on a phone often struggle to recall specific facts. Ask “Where was Leonardo da Vinci born?” or “When was Liverpool Football Club founded?” and you might get a confident, incorrect answer.

These are often the first questions people ask to see if they can trust a model. Get one wrong, and that trust is hard to earn back.

Larger models struggle with this too. One common way to help is web search: look up the question, read the results, and use them to write an answer.

But that means sending the question to an external service. We wanted to see if we could solve this without relying on the internet.

Our starting point was a hunch: perhaps a small slice of the internet could answer a lot of everyday questions.

Wikipedia seemed like a good place to start. The next step was to make it practical to download and search on a phone.

This was easier said than done. The full English Wikipedia is huge. We needed something smaller.

As we explored, a few things stood out:

• Simple English Wikipedia has shorter articles.
• Page views on English Wikipedia give us a useful signal of what people look up.
• Many basic facts appear in the opening paragraph of an article.

So we made two datasets:

• The opening paragraphs of Simple English Wikipedia articles.
• The opening paragraphs of 250,000 commonly visited English Wikipedia articles.

We built search indexes and packaged them with the text. Ensu uses RAG to search the indexes, add relevant passages to the model’s context, and respond to your question.

We added Wikibooks too, and brought these resources together as Ensu Packs. On mobile or desktop, open Settings → Ensu Packs to download and enable one. Answers that use a Pack include source attribution. Learn more.

This improved results on factual questions, particularly for the smaller models. We tested across 500 questions. Here is the percentage answered correctly, with and without Packs:

We would like to add Packs for more subjects. There are a couple of challenges:

• Finding good sources we can redistribute.
• Keeping searches fast and useful when several Packs are enabled.

While we work on those, we looked at another way to extend what we had built.

Chat with Your Notes

Some of Ensu’s use cases involve conversations that people don’t want to share with cloud services. And some of our most private thoughts already live in our notes. They felt like an obvious place to extend what we had built for Packs.

So that is what we did.

Open Settings → Your Notes and add a folder of Markdown files. Ensu indexes your notes locally. When a relevant question comes up, it finds the right passages using the same approach we built for Packs.

Edits to your notes are picked up automatically, typically within 5 - 10 minutes while the app is open. The feature works on mobile and desktop. Learn more.

What’s next

We are improving how Ensu handles long conversations within the limited context windows of small models.

After that, we plan to add basic tool calling and support for skills. We think this will give us more of the pieces we need to make Ensu useful across a wider range of tasks.

Over time, we want Ensu to choose the right skills, tools, and resources - including Packs and your notes - for what you are trying to do. We are eager to get there, and then build on top of it.

If you have tried these features, or have thoughts on where Ensu should go next, come talk to us on Discord.