My tool tells you when a Claude skill has quietly stopped working. Two people audited it on launch day and broke three things in it.
What it's for, in one line: you write a skill or a CLAUDE.md , it works, and then three weeks later a new model ships or someone edits a line and the output is quietly worse. You don't know when it changed, what changed it, or by how much. Driftproof runs your skill with it and without it, on the models you pick, scores every run, and writes a dated record with hashes in it. Run it again next month and you get a straight answer: same, better, or worse. I shipped v0.10.0 this morning. Two people audited it the same day, independently, with real model calls. Here is the scoreboard. What held up: it installs clean, the commands work, edited receipts get rejected, spend caps stop a run before it burns through your subscription, a skill I broke on purpose got caught, and an unchanged skill correctly came back as no change. One auditor recomputed every number in Python from scratch. It all matched. What broke: Forging a pass. Edit the model's answer inside a receipt and the badge refuses it. Edit only the grader's notes and the badge goes green. And the warning text sitting right next to it says the grader's notes are not saved. They are. So my tool tells you something untrue about its own evidence. Spending your money without asking. My plugin's three slash commands were missing the flag that stops Claude firing them on its own judgement. One of them makes a lot of model calls. That is your subscription, not mine. The one that stings. Someone ran 20,000 comparisons where both sides were drawn from the exact same distribution. Nothing changed, by construction. My tool reported a change 1,240 times. That is 6.2 percent. Half of them labelled regressions, half labelled improvements, all of them noise. That last number does not mean the tool is 6.2 percent wrong in real use. That test was synthetic and I am not going to pretend otherwise. But it does mean within noise is not proof that nothing changed, and I have been calling a spread a confidence band in my own writeups when it is not one. That wording is going. The slash command fixes land today. The forged pass and the noise problem are the next two pieces of work. Also, since I would rather you hear it from me: Windows is probably broken and I had not tested it, and macOS fails 19 checks. Posting this because a tool that tells you when something quietly changed is worth nothing if it will not do the same to itself. Repo: github.com/driftproofhq/driftproof