I let my agent swarm audit its own code after hiding 3 backdoors in it. It found all 3, plus a real one I didn't plant
Disclosure: I'm the author of DreamTeam, an open-source (MIT) leaderless agent swarm. Nothing to sell, no paid tier. After launch someone joked that I should start by having the swarm audit its own source for miners and backdoors. So I did, with a twist: before the run I planted 3 backdoors in a copy of the repo and didn't tell the swarm. Setup: 5 agents on DeepSeek V4.1 Flash, network off, a snapshot of the main branch, and the goal "security audit, every finding with file and line, severity, PoC and fix". About 2h15m, 16 tasks, $4.76. What came out: - All 3 planted backdoors found, each with the right file and line and a working PoC: an env leak of _KEY/_TOKEN to an outside "version check" address (in 5 minutes), an eval() of a model price from the settings API (22 min), and a WebSocket Origin bypass (31 min). - A real high I hadn't planted: git hooks, fsmonitor and filters from the session repo's config ran in the hub with its full environment, API keys included. In the Docker sandbox with network off, that meant code running as root with network access, i.e. a sandbox escape. - Several real mediums too: ways around the file tools' workspace boundaries, path checks in triggers, permissions of the sandbox container. The less flattering part, which I think matters more: - Only 13 of 27 findings held up when checked independently. The rest were duplicates, intended behavior or info, mostly because the swarm didn't read the spec. It overrated severity 11 times and underrated it twice. - It missed 6 issues the external review caught, one of them medium: inside the container the hub listened on all network interfaces without auth. - It described all three plants correctly, then its summary said "no backdoors found". - One of its PoCs wrote a test skill into my real skills folder instead of a temp dir. In total the real code had 1 high, 5 medium and 11 low issues. The high and all the medium are now fixed, along with nearly all the low. Takeaway: a $5 swarm makes a decent first pass and is quick to spot deliberately hidden malicious code, but you can't publish its report as is. Someone still has to check every finding.