Catastrophic Forgetting

In practice

It is why aggressive fine-tuning on a narrow domain can make the model worse on everything else. It is mitigated with LoRA (which freezes the original weights), mixed datasets, or regularized updates. Always evaluate with a "general" test set on top of the domain-specific one.