Skip to content
AImpact
IT EN
Safety Beginner Also known as: Aggiramento delle protezioni

Jailbreak

A technique where a user talks the model into ignoring its own safety rules, for example by asking it to pretend to be a character with no restrictions.

ShareLinkedInX

In practice

Different from prompt injection: here it is the user who tries. If you offer a public LLM service this means doing red teaming, logging conversations, and running a safety classifier in cascade over responses.

Related terms

Prompt injection Alignment Red teaming

Seen in the wild

10 entries mentioning it
  1. OpenAI Safety Evaluations Hub: a public page with model safety test results
    Medium
  2. Promptfoo 0.60.0: automatic red teaming test generation arrives
    Medium
  3. Crescendo: the multi-turn jailbreak that bypasses guardrails through gradual escalation
    Medium
  4. Many-Shot Jailbreaking: safety training overridden by context length
    High
  5. NeMo Guardrails 0.8: Colang 2.0 preview and jailbreak heuristics
    Medium
  6. HarmBench: a standardized benchmark for automated red teaming and LLM defenses
    High
  7. Lakera Guard: an API against prompt injection and data leaks in LLMs
    Medium
  8. PAIR: automated LLM-vs-LLM jailbreaking in under twenty queries
    High
  9. Universal adversarial attacks on LLMs: jailbreak suffixes that transfer to ChatGPT, Bard and Claude
    High
  10. Garak: the open source vulnerability scanner for LLMs
    Medium
← All terms