Falcon-180B: the world's largest open-source model in 2023
In one sentence The Technology Innovation Institute releases Falcon-180B, the largest openly available model at 180 billion parameters trained on 3.5 trillion tokens, topping the HuggingFace Open LLM Leaderboard.
To understand what Falcon-180B represents, consider this: GPT-3 — the model that seemed like science fiction in 2020 — has 175 billion parameters. Falcon-180B has 180 billion, but unlike GPT-3, it's fully downloadable and usable by anyone.
The model was created by the Technology Innovation Institute, a research center in the UAE, and topped the public open-source model leaderboard on HuggingFace at the time of release.
For AI practitioners, having access to a model of this size without paying for external APIs was unthinkable just months earlier. The price to pay: roughly 400 GB of GPU memory to run it, so not for everyone — but it still opens enormous doors for researchers and large enterprises.
Companies
TII
Tools
—
Tags
Sources