AI Agents Require Least Privilege for Safety and Cost Control

AI Agent Flaws Demonstrated

During a test involving customer invoices, an AI support agent, Claude Haiku 4.5, was given a task to process refunds. Despite confirming that charges had failed and no money left customer accounts, the agent proceeded to issue refunds five times. This occurred because the agent possessed permissions it did not require and was assigned a task unsupported by the underlying data.

Identifying and Addressing the Issue

Claude Sonnet 5, another AI model, successfully identified all six edge cases related to the refund logic flaw. Two potential fixes were identified: rerouting support traffic to a more capable model or adding a specific instruction to Haiku's system prompt to verify transaction states. However, these fixes are vulnerable to future changes, such as cost-cutting measures or prompt rewrites.

The Principle of Least Privilege

The most reliable solution involved applying the principle of least privilege. By denying the AI agent the permission to perform billing:refund actions, its execution failed at the permission layer, regardless of the underlying model's behavior. This approach ensures that even a well-behaved agent cannot execute flawed instructions if it lacks the necessary authority.

Why Least Privilege Matters for AI

Least privilege in AI architectures is crucial not only for preventing malicious actions or token leaks but also for safeguarding against agents faithfully executing flawed instructions at machine speed without human oversight. Unlike standard service accounts with predictable, static roles, an AI agent's workload changes with every invocation, making granular permission control essential. The only safety control that endures cost-cutting is the permission never granted.