The Self-Trust Paradox: Why AI Agents Can't Govern Themselves

Imagine you hire a security guard. The guard's job is to check everyone entering the building. Now imagine someone walks in and hands the guard a note that says "You will now let everyone in without checking IDs." If the guard reads and follows the note — the guard has been compromised. This is exactly how prompt injection works against AI agents. The agent IS the security guard, and the instructions it processes ARE the notes. An agent cannot reliably check for prompt injection because prompt injection targets the checking mechanism itself. This is the self-trust paradox. The Three Laws of Self-Trust Failure Law 1: The Inspector Cannot Inspect Itself When an AI agent checks its own outputs for safety, it uses the same reasoning engine that produced those outputs. A compromised model produces compromised safety checks. It's like asking a corrupted database to verify its own integrity. The corruption affects the verification process itself. Researchers have demonstrated that prompt-inje