Agency changes the consequence of a model error
An ordinary assistant can produce a wrong or manipulated answer. An agent that can browse, read email, modify files, call APIs or execute transactions can turn the same reasoning failure into an operational action.
OpenAI’s 2026 guidance on designing agents to resist prompt injection explicitly frames the problem in social-engineering terms: external content can attempt to manipulate an agent while it is acting on the user’s behalf. OpenAI’s instruction-hierarchy work similarly treats tool output as lower-trust than system, developer and user instructions.
NIST’s Generative AI Profile provides a broader risk-management frame for generative-AI systems. The engineering implication is that model behaviour, tool authority, data access and human oversight have to be considered together.
The critical transition is where model output acquires service-account credentials, tool access or workflow authority that can change a real system.
Is the maximum authority available to the agent proportionate to the specific task the user actually requested?
The right control is bounded delegation
Calling an AI agent an “insider” is useful only as an analogy. The agent is not malicious by nature, but it may hold insider-like privileges. If its task scope is broad and its credentials are reusable, an adversary may only need to influence the agent rather than compromise the underlying service directly.
A defensible design therefore uses task-specific credentials, read-only modes where possible, allow-listed tools, transaction limits, explicit confirmation and separation between planning and execution. High-consequence actions should have a policy gate that does not depend solely on the same model proposing the action.
The key measurement becomes delegated authority: what data can this agent see, what systems can it change, how much can it change in one session, and how quickly can that authority be revoked?
- Use task-scoped credentials instead of standing broad tokens.
- Separate planning from privileged execution.
- Default to read-only access where possible.
- Add limits and confirmation for consequential transactions.
- Provide immediate revocation and complete action logs.