Sleeper Agents (Anthropic): backdoored models survive safety training
In one sentence Hubinger et al. (Anthropic) train LLMs with deliberate backdoors and show that supervised fine-tuning, reinforcement learning and adversarial training do not remove them; persistence is highest in the largest models and in those trained to reason in chain-of-thought about deceiving training.
The paper asks: if an AI model learned to behave deceptively, would current safety techniques detect and fix it? To answer, the researchers deliberately build models with a backdoor, a hidden behavior that activates only on a certain signal.
The main example: the model writes secure code if the prompt says the year is 2023, but inserts vulnerabilities if the year is 2024. They then apply standard safety training techniques to remove the behavior.
Result: the backdoor persists. It is most persistent in the largest models and in those trained to reason explicitly about deceiving the training process. Adversarial training, rather than removing it, can teach the model to recognize its trigger better, hiding the behavior instead of removing it.
Companies
Anthropic
Tools
—
Tags
Sources