Mapping the Mind: Anthropic extracts millions of interpretable features from Claude 3 Sonnet
In one sentence On 21 May 2024 Anthropic published interpretability research in which, by applying dictionary learning to a middle layer of Claude 3 Sonnet, it identifies millions of features matching concepts such as the Golden Gate Bridge, code bugs or sycophancy, and shows that amplifying them changes the model's behavior.
How does a large language model really work internally? For years the honest answer was: we do not know. On 21 May 2024 Anthropic's interpretability team published a result that starts to change that.
By applying a technique called dictionary learning to a middle layer of Claude 3 Sonnet, the researchers isolate millions of "features": recurring patterns of neuron activations that correspond to recognizable concepts. There are features for entities such as San Francisco, Rosalind Franklin or lithium, and for abstract concepts such as bugs in code, gender bias in professions or keeping a secret. The Golden Gate Bridge feature fires both on text in several languages and on images of the bridge.
The features are not mere correlations: when the Golden Gate Bridge feature is amplified, Claude steers almost every conversation toward the bridge and even claims to be the bridge itself.
Companies
Anthropic
Tools
Claude 3 Sonnet
Tags
Sources