The attack works because instructions and data share a language
Prompt injection places malicious instructions inside content the model is asked to read, such as a webpage, document or tool result. OpenAI describes prompt injection as an industry-wide frontier security challenge and has developed instruction-hierarchy training to help models distinguish higher-trust instructions from untrusted content.
The problem becomes materially more serious when the model can call tools. A hidden instruction in retrieved content is no longer only capable of changing text output; it may try to redirect browsing, disclose information or influence an action the user never requested.
OWASP similarly classifies prompt injection as a leading risk for LLM applications. The recurring architectural mistake is allowing retrieved content to cross from the data plane into the control plane without an independent policy decision.
The dangerous transition occurs when untrusted content stops being data for a model and starts influencing an action-capable authority path.
If the model follows a malicious instruction in retrieved content, what independent control prevents it from exercising unintended authority?
Model robustness is necessary but not sufficient
Training models to respect instruction hierarchy improves resistance, but system architecture still needs least privilege. An agent reading a public webpage should not automatically inherit access to sensitive mail, internal documents or consequential actions simply because those tools are available in the same session.
Strong designs separate untrusted observation from privileged action, scope tool permissions to the current task, require confirmation for consequential changes, and validate outputs before another system executes them.
The security objective is therefore not “detect every malicious prompt.” It is to make sure that a successful prompt injection still encounters independent boundaries before it can convert text into authority.
- Label retrieved content as untrusted input.
- Scope tools to the minimum task requirement.
- Require confirmation before consequential actions.
- Keep sensitive data inaccessible to unrelated browsing tasks.
- Log tool calls and the content that influenced them.
