Build a Basic AI Agent From Scratch: Security II

Previous parts of Build a Basic AI Agent From Scratch:Basic AgentToolsLong Task PlanningHuman in the Loop & Security

You can find and clone this code in this blog series' Github repo.

In the previous part we gave our agent a basic safety model: permission modes, an acceptEdits trust boundary, and an ask_question tool so the agent could stop and clarify before doing something risky. That was enough to keep the agent from running wild on your machine, but it was only the first layer of defense.

These measures ultimately put the burden of security on the human instead of the machine, since the machine cannot be trusted. In many cases, this won't be enough. A human can be wrong, or they can glance over security issues because they are tired, or simply don't care. Once a tool call is approved by the human, the agent is free to run around and do all the damage its host allows it to do.

In this part we will start closing the gaps left open by human in the loop. We will move tool execution into a Docker sandbox so a runaway command can only touch the project directory, add prompt-injection defenses so the model stops trusting tool output as instructions, and validate every tool input against its schema before it runs. The remaining controls — resource and cost limits, secret scrubbing, audit logging, and a kill switch — are covered in the next part.

A Checklist-Driven Approach

Before writing code, it helps to lay out everything a production-grade agent harness should defend against. The codebase ships a small checklist that captures the threat model in six sections:

Prompt Injection Defense: delimit context, treat external data as data, re-validate intent

Tool Permission Gating: least privilege, destructive-action confirmation, scoped params

Input/Output Validation: validate input against schema, sanitize outputs

Loop & Resource Controls: iteration caps, token budget, timeouts, cost circuit breakers

Secret & Credential Management: no secrets in prompts,…

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论