Small Model Rampages and Safety

Now that I have an agentic AI system set up (I wrote my own local MCP server) I can test different models on it. However, it became clear in testing that occasionally a model will make a "bad" tool call—it will go on a rampage and start deleting files or inserting data incorrectly.

I spent some time working on making safety checks to prevent bad agents from ruining my data. I have backups of course, but it is better not to need to restore from backups (and more convenient).

I tested a tiny model from Hugging Face and it did not seem to understand what my MCP server was all about.

• It went on a rampage and started making bizarre tool calls.

• The safety checks I previously added prevented any data corruption.

It seems that if you are going to try to use small models, a lot of safety checks are helpful. With tiny models (less than 1 gigabyte) a model might not be able to predict tokens based on instructions well. So having logic that checks whether a tool call is likely intended and correct is important.