My app’s AI kept making things up about what people wrote. Fixing it became the whole release.

Canto is a reading journal I've been building on nights and weekends for six months. When you finish a book, it asks you a few questions, you write, and it writes back. It uses AI for the questions and the replies. This release, I read through the replies it had written about my own reading, and found it making things up. Not wildly, which would have been easy to catch. Plausibly: - It said I'd "come out of" another book "talking about the back half." I never wrote that. - It put phrases in quote marks that weren't my words. Some were close paraphrases, some were its own phrasing presented as mine. - I wrote about book 5 of a series after writing about book 6. It read my book 6 entry as where I was after book 5, and told me I was "heading into the final book" when I'd already finished it. - It told me what kind of reader I am. Confident, well written, and not based on anything I'd said. None of it was obviously broken. That was the problem. It sounded insightful, and a reader has no easy way to tell an insight from an invention. What I changed: - Quote marks are only for your exact words. Every quoted span is checked against what you actually wrote, and anything that isn't verbatim loses its quote marks. - Earlier entries carry the date you finished each book, not the date you wrote about it, so it can't confuse reading order. - If a reply borrows wording from an earlier entry without citing it, it gets rewritten. - The questions stopped leaning on a book's reputation ("this book is known for...") and started from what you actually saved or said you hoped to find. I tested every change against my own past entries before shipping it, because the failure mode only shows up when you know what you really wrote. The app: when you finish a book, you answer a few questions or write freely, and Canto writes a short reply about what you said. Everything is kept by book in a journal, so you can read back what any book meant to you later. No feed, no followers, nothing public. Honest numbers: 48 real accounts after six months, about 19 real installs a month. Two thirds of people who sign up write at least one reflection. Getting them to come back for a second book is the hard part. If you're building with AI, I'm curious how you catch the plausible-but-wrong stuff. And if you keep track of what you read, I'd love to know what would make you come back after the first book. App Store: Canto: Book Tracker & Journal