Toolformer: the LLM that learns to use tools on its own

Until now, teaching a language model to use external tools — like a calculator or search engine — required explicit training with thousands of hand-annotated examples. Expensive and inflexible.

Meta AI proposes a different approach: the model generates its own API calls during pre-training, checks whether the tool's output improves its prediction, and keeps only the useful calls. No human supervisor needed.

The result is a model that fluidly uses five different tools, significantly improving on numerical computation, factual questions, and temporal reasoning — outperforming GPT-3 on several tasks despite being much smaller.