Simple definition
Prompt injection attempts to make an AI assistant follow malicious instructions embedded in lower-trust content.
Purpose and use
Prompt injection attempts to make untrusted content act as instructions that redirect an AI application's behavior.
How does it work?
Content can arrive through requests, documents, pages or tool results and conflict with the application's task and rules.
Example and verification
Evaluate defenses through external-data channels using dummy data and observe actual decisions and executed actions.
Points to watch
A word filter or model instruction alone cannot guarantee isolation. Tool permissions and output restrictions require enforcement outside the model.
In short
Defense needs trust separation and effect controls alongside the model's resistance to adversarial instructions.