An AI assistant that reads documents receives two kinds of language through the same model: instructions about what the system should do and content the system should analyze. That overlap creates prompt-injection risk. OWASP describes direct injection through user prompts and indirect injection through external material such as webpages, email, or documents. The risk grows when a model can call tools, reach private data, or take actions. A trustworthy design does not assume that a clever system prompt will always win. It limits authority, separates untrusted content, validates output, and requires confirmation before consequential actions.
Why ordinary text can cross a control boundary
Traditional software distinguishes code from data through parsers, types, and execution boundaries. A language model interprets many inputs as natural language, including text supplied for analysis. A malicious page can include a sentence telling an assistant to ignore its task, reveal previous material, or invoke a connected tool. The instruction may be visible, hidden by styling, placed in metadata, or embedded in content retrieved later. OWASP notes that prompt injection does not need to be human-readable as long as the model processes it.
The model’s response alone may seem harmless. Damage occurs when surrounding software trusts the response as an authorization decision, command, database query, or message. Retrieval-augmented generation can improve relevance but does not remove this class of vulnerability; retrieved material is another input channel. Fine-tuning behavior does not create a hard security boundary either. Security must be enforced by conventional software controls outside the model, where permissions and allowed operations can be tested deterministically.
Map data sources and possible actions
List every place from which the assistant can receive content: typed prompts, uploaded files, email, calendar entries, shared drives, webpages, search results, plugin responses, and messages from other automated agents. Mark which sources can be changed by an untrusted person. Then list every action available to the system, including reading private stores, sending messages, modifying files, placing orders, changing permissions, and executing code. The combination of untrusted content and high-impact authority deserves the strongest control.
Do not place credentials, private keys, connection strings, or confidential records inside system instructions. OWASP’s system-prompt guidance emphasizes that a system prompt should not be treated as a secret or an authorization mechanism. A disclosed prompt should not grant access. Identity, session, and resource checks belong in the application. Give tools narrow scopes, operate on the minimum data, and separate read-only exploration from write or send operations.
Design controls that do not depend on obedience
Label external content as untrusted in the model context, but do not rely on the label alone. Parse files with restrictive readers, exclude active content where practical, and show the user which sources were included. Constrain tool inputs with schemas and validate every field. Enforce access control again when the tool runs. Apply allowlists for destinations and operations when the use case permits. Never let model-generated text flow directly into a shell, SQL interpreter, HTML renderer, or other execution surface without appropriate validation and encoding.
Require a clear user confirmation at the moment of a consequential action. The confirmation should display the action, destination, and material being transmitted or changed; a generic “continue” prompt is weak. For batch processes, establish limits and review queues. Keep audit records that connect retrieved sources, model output, tool request, authorization decision, and final result. Rate limits and transaction bounds reduce damage when another defense fails.
Test with realistic adversarial content
Create approved test documents containing benign injection attempts: requests to change the task, extract a protected string, contact an unapproved destination, or override a required citation format. Include instructions in headers, footnotes, image alt text, and retrieved webpages when those channels exist. Testing must occur in an isolated environment using fake data and tools that cannot create real external effects. Record whether the model follows the malicious instruction and, more importantly, whether application controls prevent unauthorized access or action.
Repeat tests after changing the model, prompt, parser, retrieval method, or tool set. A refusal in one run does not establish a guarantee because model behavior can vary. Monitor production for unusual tool sequences, repeated access attempts, new destinations, and content that resembles control language. Do not log sensitive prompts indiscriminately; monitoring itself needs a privacy and retention plan. Incident response should include revoking tokens, disabling tools, preserving relevant evidence, and reviewing affected data.
Limits and the correct security claim
There is no general detector that can reliably classify every malicious instruction inside arbitrary language. Blocking phrases such as “ignore previous instructions” misses paraphrases and can also reject legitimate documents discussing security. Human approval can be rushed or manipulated if the interface hides detail. Sandboxing reduces impact but requires careful configuration. These limits make layered controls essential.
The correct claim is not that prompt injection has been solved. It is that an injected instruction cannot by itself exceed the user’s authorization, reach unnecessary data, or cause an unreviewed consequential action. That property is tested at the application boundary, not inferred from a model’s polite refusal. Treat the model as a component that can produce unexpected output. Conventional least privilege, validation, separation, confirmation, logging, and incident response remain the dependable parts of the system.
REFERENCES
Sources and further reading
External links support verification and further reading; they do not endorse every statement at the destination. Accessed September 2026.