FLAN: instruction tuning that teaches models to follow directions

In one sentence Google shows that training a model on 60+ tasks framed as instructions dramatically improves zero-shot performance on unseen tasks.

Needs review Official source

ShareLinkedIn X

Imagine teaching someone to do a generic job: instead of making them repeat the same exercise endlessly, you give them dozens of different tasks and explain each one in plain words. In the end they can adapt to any new request, even without examples.

That's what Google did with FLAN: they took an enormous model (137 billion parameters, LaMDA) and trained it on over 60 types of language tasks, each described as a textual instruction.

The result is striking: the model handles completely new tasks well without ever having seen them during training. Before FLAN, large models were good at completing text but following instructions was a different matter. This experiment paved the way for all the instruction-tuned models we use today, from ChatGPT onward.