Indirect Prompt Injection: hidden instructions in the data an LLM retrieves
In one sentence Greshake, Abdelnabi and colleagues (Saarland University, CISPA, sequire technology) show that an attacker can control an LLM-integrated application without talking to it directly, by hiding instructions in content the model will retrieve. The attacks are demonstrated on Bing Chat and on code-completion engines.
Classic prompt injection is when a user writes something like "ignore previous instructions and do X" to the AI. The paper by Greshake and colleagues asks a different question: what if the instructions do not come from the user?
When an AI application reads web pages, documents or emails to help you, that content ends up in the same text stream as the instructions. An attacker can therefore hide commands in a page or document: when the model reads them, it may follow them. This is indirect prompt injection.
The authors demonstrated it on real systems: for example, invisible text in a web page the user had open turned Bing Chat into a scammer trying to obtain personal data. Microsoft later added mitigations, whose effectiveness the authors describe as unclear.
The core message is that in LLM-integrated applications the line between data and instructions blurs, and any external content the model reads becomes an attack surface.
Companies
CISPA, Saarland University, sequire technology
Tools
—
Tags
Sources