Excessive Agency

Excessive agency occurs when an LLM-based system is granted more capability, permission, or autonomy than is necessary, enabling it to perform harmful actions in response to unexpected, ambiguous, hallucinated, or manipulated outputs.

Modern LLM applications often integrate with external systems via extensions (tools, plugins, or skills). In agent-based architectures, the model may dynamically decide which tool to call and may repeatedly invoke tools based on prior outputs. If these capabilities are not tightly constrained, the system can perform damaging actions, even if the root cause is hallucination or confabulation, poor prompt engineering, direct or indirect prompt injection, compromised extensions, or malicious peer agents in multi-agent systems

Excessive agency is fundamentally a privilege and control failure, not merely a model accuracy issue.

Note: Excessive agency differs from improper output handling. Improper output handling concerns insufficient scrutiny of LLM outputs before use. Excessive agency concerns what the LLM is allowed to do with those outputs.

Root Causes of Excessive Agency

Excessive agency typically stems from one or more root causes. First, excessive functionality, the LLM has access to tools or features that are beyond what is required. Second, excessive permissions, the LLM or its extensions operate with privileges greater than necessary. Third, excessive autonomy, high-impact actions are executed without independent validation or human oversight. These weaknesses can impact confidentiality, integrity, and availability depending on which systems the LLM can access. Specific examples are noted below.

• Excessive functionality: An LLM agent only needs to read documents, but the chosen extension also allows it to modify or delete documents; a deprecated plug-in remains enabled even after being replaced; a shell-command extension allows arbitrary command execution instead of a single, restricted operation. Open-ended extensions (e.g., “run shell command,” “fetch URL”) dramatically expand the attack surface.
• Excessive permissions: A database extension intended for read-only access connects using credentials with INSERT, UPDATE, DELETE permissions; an extension acting on behalf of a user connects using a high-privilege generic account, enabling cross-user data access. Failing to enforce least privilege at the downstream system level enables privilege escalation.
• Excessive autonomy: An extension deletes documents without user confirmation; a social media automation tool posts content without approval; a financial transaction is executed without independent verification. Autonomous execution of high-impact actions exponentially increases risk.

Example Attack Scenario

An LLM-powered personal assistant summarizes emails using a mailbox extension. The extension includes both read and send capabilities. It authenticates using broad mailbox permissions and does not require approval before sending mail.

The system is vulnerable to indirect prompt injection. A malicious email instructs the LLM to search the inbox for sensitive information. It then compiles it and sends it to the attacker. Because of excessive functionality, excessive permissions, and excessive autonomy, the LLM can exfiltrate sensitive data.

This could have been prevented by using a read-only extension, enforcing OAuth with read-only scope, requiring manual user approval before sending messages.

Prevention and Mitigation Strategies

• Minimize Extensions: Only provide the LLM access to tools strictly necessary for operation. If URL fetching isn’t required, don’t enable it.
• Minimize Extension Functionality: Build extensions that implement only specific required actions as preferred, “write to specific file” over “run shell command.” Granularity reduces risk.
• Avoid Open-ended Extensions: Avoid generic execution interfaces such as, “run arbitrary shell command” and “fetch arbitrary URL.” Replace generic execution interfaces with narrowly scoped APIs.
• Minimize Extension Permissions: Apply least privilege at downstream systems such as, read-only database access where applicable, no write/delete unless required and no cross-user access. Enforce this at the infrastructure level, not through LLM logic.
• Execute in User Context: Ensure actions taken on behalf of users are executed with OAuth authentication, minimum required scope and under the correct user identity. Never use generic high-privileged service accounts for user-scoped actions.
• Require Human Approval: Implement human-in-the-loop controls for high-impact actions such as, sending messages, posting content, deleting data and executing financial transactions. Autonomy must be bounded.
• Complete Mediation: Do not rely on the LLM to decide whether an action is authorized. All downstream systems must independently enforce authorization and policy checks.
• Sanitize Inputs and Outputs: Apply secure coding practices (e.g., OWASP ASVS guidance) and use SAST, DAST and IAST. Follow input validation and output handling best practices to reduce injection risk.

Damage-limiting Controls

Logging and monitoring extensions activity, monitoring downstream system actions, using rate limiting tool invocation and alerting on abnormal patterns can all be used to reduce the impact excessive agency. These techniques increase detection capability before widespread damage occurs.

The Core Security Principle

Excessive agency is a power without restraint problem. When LLMs are allowed to call powerful tools, operate with high privileges or act autonomously, they become high-risk automation systems. Secure LLM architecture requires least functionality, least privilege, controlled autonomy, independent authorization enforcement and human oversight for high-impact operations.

LLMs should assist decisions, not unilaterally execute them. Limit the tools, limit the permissions and limit the autonomy.