Crescendo: the multi-turn jailbreak that bypasses guardrails through gradual escalation
In one sentence Mark Russinovich, Ahmed Salem and Ronen Eldan of Microsoft publish Crescendo, a multi-turn jailbreak that starts with innocuous questions and leverages the model's own answers; successfully tested on ChatGPT, Gemini Pro, Gemini Ultra, LLaMA-2 70b Chat and Anthropic Chat.
Language model guardrails are trained to recognize and block explicitly dangerous requests. But what happens when no single request looks dangerous?
The Crescendo technique, described by Microsoft researchers, starts with a general question on the topic and then gradually escalates, referencing answers the model itself has already given. Step by step the conversation reaches where a direct request would have been refused.
Because the messages look innocuous, filters that look for explicitly malicious content in inputs struggle to recognize the attack.
The authors report high success rates across all evaluated models and tasks, including ChatGPT, Gemini and Anthropic's chatbot.
Companies
Microsoft
Tools
Crescendo, Crescendomation
Tags
Sources