Category
35 entries
DeepSeek releases V4 Preview as open source: V4-Pro (1.6T total, 49B active) and V4-Flash (284B total, 13B active). Native 1M-token context, hybrid CSA+HCA attention cutting KV cache by 90%.
Mistral releases Small 4, unifying Magistral (reasoning), Pixtral (multimodal vision), and Devstral (agentic coding) into a single open-weight model, simplifying the deployment stack.
Mistral AI announces a new flagship with extended reasoning, open weights for the research variant. Europe's answer to DeepSeek R2 and the US reasoning models.
DeepSeek ships R2, successor to R1: more efficient step-by-step reasoning, open weights, contained training cost. Fresh pressure on closed reasoning models.
Alibaba ships Qwen 3: 8 models from 0.6B to 235B params (2 MoE + 6 dense), all with switchable thinking mode. Apache 2.0 license. Repositions Qwen as the best open weight.
Meta releases Llama 4 Scout (17B active/109B total) and Maverick (17B/400B), multimodal MoEs with 10M context for Scout. Behemoth (2T) in training. Benchmark claims contested by the community.
DeepSeek releases a DeepSeek-V3 update (685B param MoE, 37B active) under MIT license. Performance close to Claude 3.7 Sonnet on coding, training cost estimated 20x lower.
Chinese startup DeepSeek releases R1, a reasoning model with MIT-licensed open weights. Performance on par with OpenAI o1, API pricing $0.55/$2.19 per 1M tokens (vs o1 $15/$60). Nasdaq AI loses $1T in two days.
DeepSeek publishes V3, MoE 671B (37B active), competitive with GPT-4o and Claude 3.5 Sonnet. Training: 2.788M H800 GPU-hours, claimed cost $5.6M. Changes the 'frontier = billions' narrative.
Meta releases Llama 3.3 70B Instruct: same parameter count as 3.1 70B but reported performance close to 405B thanks to a new post-training pipeline — no new base model.
Allen Institute (AI2) releases Tülu 3: 8B/70B family with the first truly open post-training pipeline (code, data, recipes, eval), beating Llama 3.1 Instruct using only Meta's base.
Meta releases Llama 3.2 in 4 sizes: 1B and 3B for edge/mobile, 11B and 90B multimodal (vision). First time Meta seriously enters open multimodal + on-device.
Alibaba releases Qwen 2.5: 7 sizes (0.5B–72B), updated tokenizer, specialized Coder and Math variants, positioning the family as the open multilingual and code-strong reference.
Meta releases Llama 3.1 405B under commercial license: for the first time an open model directly competes with GPT-4 and Claude 3.5 Sonnet on benchmarks, with 128K context.
Google releases Gemma 2 (9B and 27B), a second-gen open family with Gemini-derived architecture, soft attention capping, knowledge distillation, and class-leading performance in the <30B range.
DeepSeek releases V2: 236B-total / 21B-active MoE with Multi-head Latent Attention (MLA), drastically cuts KV cache, slashes Chinese API prices by 90%, and ignites a price war.
Meta releases Llama 3 in two initial sizes (8B, 70B). Trained on 15T tokens, improved tokenizer, 8K context. The 70B Instruct competes with Claude 3 Sonnet and Gemini 1.5 Pro on many benchmarks.
Mistral releases Mixtral 8x22B under Apache 2.0, a 141B-total / 39B-active MoE with 64k context and an optimized tokenizer, the first open-weight model to truly rival Llama 2 70B in production.
Databricks releases DBRX, an open-weights Mixture-of-Experts with 132B total parameters (36B active per token), beating Llama 2 70B on many benchmarks at lower inference cost.
Google releases Gemma 2B and 7B, open-weight models derived from Gemini research. For the first time Google competes directly with Llama and Mistral on open ground.
Mistral drops Mixtral 8x7B via magnet link with no warning: SMoE with 8 experts of 7B, 13B active params out of 47B total. Performance matches/exceeds GPT-3.5. Apache 2.0.
Mistral AI (Paris), a three-month-old startup founded by ex-Meta/DeepMind researchers, releases Mistral 7B under Apache 2.0. Beats Llama 2 13B on most benchmarks with half the parameters.
Meta releases Llama 2 (7B, 13B, 70B) under a license that allows commercial use up to 700M MAU. For the first time a serious LLM is genuinely deployable to production without depending on an API.
Stability AI releases StableLM 3B and 7B under CC BY-SA 4.0, trained on 1.5T tokens. Open response to closed models, but quality still trails LLaMA.
Meta releases LLaMA in four sizes (7B, 13B, 33B, 65B), available to researchers on request. One week later, the weights leak publicly.
Hugging Face releases diffusers, a modular Python library for diffusion models — text-to-image, audio and beyond. It quickly becomes the de facto standard.
The BigScience collective releases BLOOM, a 176-billion-parameter model trained on 46 human languages and 13 programming languages, under an open RAIL license.
Meta AI releases OPT-175B, a language model comparable in size to GPT-3, with weights available to researchers and a public training logbook.
EleutherAI releases GPT-J, a 6B-parameter model trained in JAX on TPUs, performance comparable to GPT-3 Curie, shipped under Apache 2.0.
EleutherAI publishes The Pile, an 825 GB dataset built from 22 diverse sub-datasets — the base for GPT-Neo, GPT-J, Pythia and much of the early open source ecosystem.
EleutherAI releases GPT-Neo 1.3B and 2.7B, open source language models trained on The Pile — the first serious attempt to replicate the GPT-3 architecture with public weights.
EleutherAI releases The Pile, an 825 GB composite text dataset curated from 22 different sources (arXiv, GitHub, PubMed, books, StackExchange…), designed for pre-training large open-source language models.
HuggingFace releases Transformers 3.0 with the Rust-based tokenizers library (up to 100× faster), new NLP pipelines, and tighter Model Hub integration, cementing the de facto standard for using pretrained models in Python.
Connor Leahy, Sid Black, and Leo Gao found EleutherAI on Discord with the goal of replicating GPT-3 and releasing models, code, and datasets in the open, kicking off projects like GPT-Neo, GPT-J, and The Pile.
Facebook AI Research consolidates fairseq as the reference sequence-to-sequence framework: it adds modular support for BART, RoBERTa, mBART, wav2vec and becomes the primary codebase for FAIR's 2020 models.