BLOOM 176B: the first truly open large multilingual LLM
In one sentence The BigScience collective releases BLOOM, a 176-billion-parameter model trained on 46 human languages and 13 programming languages, under an open RAIL license.
While OpenAI keeps GPT-3 closed and Meta opens OPT only to researchers, a group of a thousand volunteers from 70 countries — the BigScience project led by Hugging Face — builds a same-scale model in public and releases it under an open license.
It's called BLOOM, 176 billion parameters, trained on a French supercomputer (Jean Zay) for four months. The difference from GPT-3: it understands 46 human languages, not just English, and 13 programming languages.
It doesn't win on performance, but it's another step toward AI not controlled by a single company. Its "Responsible AI License" (RAIL) tries to ban harmful uses while staying open.
Companies
BigScience, Hugging Face
Tools
BLOOM-176B
Tags
Sources