High Foundation Models · 1 min read
Gopher 280B: DeepMind enters the large language model race
In one sentence DeepMind publishes Gopher, a 280B-parameter dense model, with a 152-task study: it beats the state of the art on 100 of 124 comparable tasks, but scale helps little on logic and math.
Reading level
DeepMind, until then best known for AlphaGo and AlphaFold, unveils Gopher, a language model with 280 billion parameters.
The model comes with a study across 152 tasks: Gopher beats earlier models in most of them. Scale helps a lot with reading comprehension and fact checking, much less with logical reasoning and math.
On the same day DeepMind also publishes a paper on the ethical and social risks of language models and the RETRO paper, a model that looks things up in a text database.
Companies
DeepMind
Tools
Gopher
Tags
DeepMindGopherScalingBenchmarkPaper
Sources