Landmark Foundation Models · 1 min read
GPT-3: the paper that opens the scaling-laws era
In one sentence OpenAI publishes 'Language Models are Few-Shot Learners' and shows that at 175B parameters a model learns new tasks from a handful of examples in the prompt.
Reading level
OpenAI introduces a huge language model, GPT-3. The novelty isn't just size: the model learns to do new tasks by looking at a few examples written in the conversation, without being retrained.
Before, teaching a model a specific task required a dedicated training phase. GPT-3 shows that beyond a certain scale, the model "gets" what you want from a few quick examples.
For software people it's a paradigm shift: instead of training a model for each problem, you write a prompt.
Companies
OpenAI
Tools
GPT-3
Tags
OpenAIGPT-3Few-shot LearningTransformerScaling Laws
Sources