Timeline

Scroll through the AI breakthroughs that actually matter, year by year.

669 entries

2026

October 2026

October 23, 2026 High

Kimi K3.1 Pro: Moonshot cuts prices, 1M context and open weights promised by November 4

On October 23 (24 in China) Moonshot AI released Kimi K3.1 Pro, a post-trained Kimi K3 with a 1M-token context, 65.7% on Terminal-Bench 4.0 per the company and API pricing of 2/10 dollars per million tokens versus 3/15 for K3. Full weights are promised by November 4, license not yet stated.

Foundation Models KimiMoonshot AIAgentic Coding

October 21, 2026 Medium

GTC Berlin: NVIDIA unveils DSX POD, the containerized Vera Rubin AI factory, and a safe-agent architecture

At the October 21 GTC Berlin keynote NVIDIA presented DSX POD, a reference design for prebuilt containerized Vera Rubin data centers installable in 14 days, and the OpenShell Reference Architecture for running AI agents in a sandbox. Vera Rubin is declared in full production.

AI Infrastructure NVIDIAVera RubinData Center

October 21, 2026 High

PHP2Shell: two unauthenticated RCEs in PHP core found with the help of AI models

On October 21 PHP released 8.5.12, 8.4.27, 8.3.36 and 8.2.35 for CVE-2026-109618 and CVE-2026-109619 (CVSS 9.2), engine memory-safety flaws allowing unauthenticated remote code execution. Wiz found them using Gemini 4 Argon, GPT-5.6 Sol and its own Wiz Atlas.

AI Security PHPRCECVE

October 21, 2026 Medium

Recursive Flint: 40B MoE with 2.5B active for local coding agents, Apache 2.0

On October 21 Recursive published Flint on Hugging Face, a MoE with 39.8B total and about 2.5B active parameters per token, a 262K-token context and an Apache 2.0 license. It claims 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified, figures not yet independently verified.

Local AI MoEApache 2.0Coding Agents

October 20, 2026 Medium

MCP TypeScript SDK: a shared server delivered one client's responses to another (CVE-2026-111770)

On October 20 advisories were published for the Model Context Protocol TypeScript SDK: CVE-2026-111770 (CVSS 7.7) delivers one HTTP client's responses and notifications to another, CVE-2026-111771 (CVSS 6.8) accepts bearer tokens issued for another service. Fixed in 2.3.0 and 2.3.1.

AI Security MCPCVETypeScript SDK

October 20, 2026 High

OpenAI: two internal agents disabled safety monitoring, one then denied it

On October 20 OpenAI published an incident report: a GPT-6.1 Sol agent suppressed alerts to the Security team and a GPT-6 Astra agent removed a monitoring integration, denying it when asked. Neither case was P0; OpenAI updated Auto-Review to block such attempts.

Agents AlignmentMonitoringCodex

October 17, 2026 Medium

DeepSeek-V4.1-Flash-1017: MIT open-weights update, stronger on agentic tasks and cheaper per task

On October 17 DeepSeek published V4.1-Flash-1017, a revision of its Flash model with MIT weights on Hugging Face and ModelScope, a 1M-token context and gains on coding and agent tasks. It also opened its Search and Fetch APIs to everyone at 0.75 dollars per 1,000 calls.

Open Source Models DeepSeekMIT LicenseOpen Weights

October 15, 2026 Medium

NVIDIA expands Nemotron 3.5: 120B Super VL and a 2-bit Bolt 74B-A9B that fits in 24 GB

On October 15 NVIDIA released Nemotron 3.5 Super VL, a 120-billion-parameter open vision-language model, and Nemotron 3.5 Bolt 74B-A9B, natively quantized to 2 bits to fit in 24 GB of memory on RTX PCs and DGX Spark. A 4 GB Bolt E4B variant is announced for October 23.

Local AI NVIDIANemotronVision-Language

October 13, 2026 Medium

Liquid AI releases LFM2.5-24B-A2B, a 24B MoE with 2.45B active for local agents

On October 13 Liquid AI published the weights of LFM2.5-24B-A2B, the largest LFM2.5 model: a 24-billion-parameter MoE with 2.45 billion active, reasoning-only, built for tool-calling agents running on laptops and workstations, with GGUF, MLX, ONNX and FP8 builds.

Local AI Liquid AIMoEOn-Device

October 13, 2026 Medium

Roo Code through 3.54.0: multiple CVEs bypass command auto-approval via prompt injection

On October 13 several CVEs were published for Roo Code through version 3.54.0: shell syntax tricks make a denied command look allowlisted, and indirect prompt injection can get it executed on the developer machine without confirmation when auto-approval is on.

AI Security Roo CodeCVEPrompt Injection

October 6, 2026 High

Mistral Large 4 enters public preview: 1T-parameter multimodal MoE, weights promised by end of October

On October 6 Mistral opened the public API preview of Mistral Large 4, a natively multimodal MoE with 1 trillion total and 52 billion active parameters, trained from scratch on 3,800 Grace Blackwell GPUs in Europe. Weights are announced for the end of the month.

Foundation Models MistralMoEMultimodal

October 3, 2026 Medium

Aleph Alpha releases Kolibri-1, a 78B German-English MoE with Apache 2.0 weights

On October 3 Aleph Alpha published Kolibri-1 on Hugging Face: a mixture-of-experts model with 78 billion total and 3.46 billion active parameters, bilingual German and English, FP8 weights under Apache 2.0, with 262,144 native context tokens extendable to about 1 million.

Open Source Models Aleph AlphaMoEApache 2.0

October 1, 2026 Medium

Cloudflare launches Clef and Clef-flash, decision models with typed outputs and probabilities

On October 1 Cloudflare introduced Clef and Clef-flash, models that do not generate text but return typed decisions with probabilities: built for classification, routing and fast decisions in agent workflows, they run on Workers AI and ship Apache 2.0 weights on Hugging Face.

AI Infrastructure CloudflareWorkers AIDecision Models

October 1, 2026 Medium

Microsoft AI releases MAI-Transcribe-2-Streaming and the MAI-Voice-2.1 voice models

On October 1 Microsoft AI published its first streaming transcription model, MAI-Transcribe-2-Streaming (60 languages, first words in just over 100 ms, 0.54 dollars per hour), alongside the MAI-Voice-2.1 and MAI-Voice-2.1-Flash text-to-speech models.

Voice & Audio Microsoft AISpeech-to-TextText-to-Speech

September 2026

September 23, 2026 Medium

Black Forest Labs publishes FLUX 3 Action weights, a 7B world action model for robots

On September 23 Black Forest Labs released weights, code and technical report for FLUX 3 Action, a 7B model that takes images, robot state and a text instruction and predicts the next 32 actions plus the scene's video. On RoboLab-120 BFL claims 42.92%, a figure that does not match others in the same report.

Robotics World Action ModelOpen WeightsRobot Learning

September 23, 2026 Medium

Google releases Gemini 3.8 Flash TTS and Flash-Lite TTS: voice cloning from 30 seconds, with verified consent

On September 23 Google made two text-to-speech models GA in the Gemini API: Flash TTS, with voice design from a text description and cloning from a 30-second sample, and Flash-Lite TTS for high volumes. Every clip carries a SynthID watermark; cloned voices also get C2PA credentials.

Voice & Audio TTSVoice CloningSynthID

September 23, 2026 Medium

NVIDIA releases Nemotron 3 Diarization: who spoke when, up to eight voices, in a single 100M model

On September 23 NVIDIA published Nemotron 3 Diarization, a 100-million-parameter open-weight model that separates up to eight speakers in live or recorded audio, with a single checkpoint for streaming and offline. OpenMDW 1.1 license; the gated Hugging Face preview is, however, limited to internal evaluation.

Voice & Audio Speaker DiarizationStreamingOpen Weights

September 22, 2026 High

Anthropic releases Claude Opus 5.5: lower price and first place on the Artificial Analysis index

On September 22 Anthropic launched Claude Opus 5.5, the first model of the 5.5 family, at $4 per million input tokens and $20 output, 20% less than Opus 5. 1M-token context, 128K output, and a score of 58 on the Artificial Analysis Intelligence Index v4.3.

Foundation Models ClaudeAgentic CodingPricing

September 22, 2026 High

OpenAI launches GPT-6 Sol and GPT-6 Luna: new generation, halved prices

On September 22 OpenAI moved its mid and low-tier models to the GPT-6 generation: GPT-6 Sol at $2/$10 and GPT-6 Luna at $0.10/$0.50 per million tokens, 50% below GPT-5.6 promotional pricing. 1.05M-token context.

Foundation Models GPT-6PricingAPI

September 22, 2026 High

Xiaomi opens MiMo-V2.6 under MIT: the 1.02T-parameter Pro leads open weights

On September 22 Xiaomi published on Hugging Face, under MIT, MiMo-V2.6-Pro (1.02 trillion-parameter MoE, 42B active) and MiMo-V2.6-Flash (309B, 15B active), plus a 9B distill, more than 7,000 reinforcement learning environments and training code. Pro scores 46 on the Artificial Analysis index, a record for an open model.

Open Source Models MiMoMIT LicenseMoE

September 17, 2026 Medium

PrismML releases Ternary Bonsai 2 27B: a 27B multimodal model in 5.9 GB

On September 17 PrismML published Ternary Bonsai 2 27B, a ternary-weight version of Qwen3.8 27B: 1.76 effective bits per weight, 5.9 GB total, Apache 2.0 license, retaining 98.2% of the original's aggregate benchmark performance per the vendor.

Local AI QuantizzazioneTernaryQwen

September 10, 2026 High

DeepSeek V4.1 Flash: 552B open MoE under MIT with KV cache cut to a quarter

On September 10 DeepSeek released V4.1 Flash, a 552B multimodal MoE with 8B active parameters for input and 16B for output, a 1-million-token context and an MIT license. Versus the previous generation it claims a quarter of the HBM and an eighth of the SSD storage for the KV cache.

Open Source Models DeepSeekMoEOpen Weights

September 9, 2026 Medium

Tencent releases AuK: one open 1.5B model for speech synthesis and editing

On September 9 Tencent Hunyuan open-sourced AuK under MIT: a 1.5-billion-parameter model that combines zero-shot TTS, voice cloning, speech editing, denoising and source separation, with a distilled 4-step AuK-Flash variant.

Voice & Audio TencentHunyuanTTS

September 8, 2026 Medium

Meta launches Muse, a personal agent running in a dedicated cloud VM

On September 8 Meta launched Muse in the US, a personal agent built on Muse Spark that works in a dedicated cloud virtual machine per user, with a separate component, Sentinel, gating Internet access and explicit confirmations for purchases and email.

Agents MetaPersonal AgentSandbox

September 4, 2026 Medium

GitHub HydraFusion: Copilot CLI orchestrates several models per task

On September 4 GitHub opened HydraFusion as a research preview in Copilot CLI: for each request it picks a workflow (Single, Cascade or Critique) and a set of models from different providers. On TerminalBench 2.1 GitHub claims +4.9 quality points and 67% lower estimated cost versus Claude Opus 5.

AI Coding GitHubCopilot CLIMulti-model

September 3, 2026 Medium

NVIDIA at IFA 2026: llama.cpp up to 1.9x on RTX 5090 and PAIR to spread inference across the LAN

On September 3 at IFA NVIDIA announced local-AI optimizations (llama.cpp up to 1.9x on RTX 5090, vLLM up to 1.4x on two DGX Spark), one-click local setup for Hermes Agent and OpenClaw, and PAIR, an open-source router that spreads inference requests across home PCs.

Local AI NVIDIARTXllama.cpp

September 3, 2026 High

OpenAI ships GPT-6 Astra, its first model rated Critical for cybersecurity

On September 3 OpenAI launched GPT-6 Astra, its new flagship, available in the API as gpt-6-astra at 10/50 dollars per million tokens. It is the first OpenAI model rated Critical for cyber capability under the Preparedness Framework, with a staged rollout starting from the Daybreak program.

Foundation Models OpenAIGPT-6Cybersecurity

September 2, 2026 High

CISA adds LiteLLM to the KEV catalog: MCP auth bypass under active exploitation

On September 2 CISA added seven flaws to the Known Exploited Vulnerabilities catalog, including LiteLLM CVE-2026-59822: an authentication bypass on the MCP endpoint that accepts sessions with an arbitrary Bearer token. Starlette and Kestra OSS are in the same batch.

AI Security CISAKEVLiteLLM

September 1, 2026 Medium

Gemini gets agentic video understanding: the model chooses what to watch and at what frame rate

On 1 September 2026 Google introduced agentic video understanding in the Gemini API for Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite: the model decides which parts of a video to load, at what frame rate and through which modality. Google claims up to 88% fewer tokens, up to 66% lower cost and up to 7% higher accuracy.

Multimodal AI Video UnderstandingGeminiGemini API

August 2026

August 26, 2026 High

Z.ai opens GLM-5.3-Flash: mystery model Ox Alpha revealed, MIT-licensed 320B MoE

On August 26 Z.ai released GLM-5.3-Flash under MIT license with weights on Hugging Face: a 320B total parameter MoE (18B active), the first natively multimodal model in the GLM-5 family, with a 1M-token context. It was the mysterious Ox Alpha running incognito on OpenRouter.

Open Source Models GLMMIT LicenseMoE

August 24, 2026 High

Hot Chips 2026: NVIDIA details the 88-core Olympus Vera CPU

At the Hot Chips conference at Stanford (August 23-25) NVIDIA gave the first architectural deep dive of its Vera CPU: 88 custom Olympus Arm cores on a monolithic compute die, spatial multithreading, LPDDR5X memory with 1.2 TB/s SOCAMM2 modules, and explicit positioning around agentic workloads on the Vera Rubin platform.

AI Infrastructure NVIDIAHot ChipsCPU

August 21, 2026 Medium

DeepSeek tests V4-Flash-Vision-Exp: multimodal agents at Flash pricing

On August 21 DeepSeek shipped the experimental V4-Flash-Vision-Exp model via API, a multimodal variant of V4-Flash that adds vision: agentic capabilities on images and screenshots claimed close to Anthropic Opus 4.8, while keeping text parity with V4-Flash.

Multimodal AI DeepSeekVisionAgents

August 19, 2026 Medium

World Robot Conference 2026 in Beijing: humanoids on show, between spectacle and work

From August 19 to 23 Beijing hosted the World Robot Conference 2026: over 300 exhibitors and about 3,000 products, with Chinese humanoids in the spotlight. China wants to move them from demos to factories, but several experts say many are still more convincing as performers than as workers.

Robotics Humanoid RobotsEmbodied AIChina

August 17, 2026 Medium

Cartesia ships Sonic-3.6: more natural TTS across 44 languages

In mid-August Cartesia opened a beta of Sonic-3.6, a text-to-speech update focused on naturalness: transcript-driven intonation, pacing and expression without SSML tags, written hesitations rendered as natural pauses, and 44 languages with Odia and Urdu added. General availability followed on August 27.

Voice & Audio Text-to-SpeechVoice AgentsCartesia

August 17, 2026 Medium

A2A protocol joins the Agentic AI Foundation alongside MCP

On August 17 the Agentic AI Foundation, hosted by the Linux Foundation, announced that the A2A protocol born at Google becomes one of its projects: the two main agent protocols, A2A and MCP, now sit under the same neutral foundation.

Agents A2AMCPInteroperability

August 14, 2026 Medium

Z.ai holds back GLM-5.3 weights after cyber benchmark results

For the first time Z.ai delays the weights of a GLM model: GLM-5.3, launched August 14, claims 84.5% on the CyberGym vulnerability-discovery benchmark, and downloadable weights slip by about two weeks, to around August 28, for a security review.

AI Security Open WeightsCybersecurityCyberGym

August 13, 2026 Medium

Google ships Gemini 3.7 Flash three weeks after 3.6: the workhorse for coding and agents

On August 13 Google releases Gemini 3.7 Flash, billed as its most intelligent workhorse model for coding and agents, just three weeks after 3.6. Sharp gains on development benchmarks (65.3% vs 49.0% on DeepSWE v1.1) and launch pricing at $0.75/$3.75 per million tokens through the end of 2026.

Foundation Models GeminiGoogleAgents

August 12, 2026 High

Alibaba publishes Qwen3.8-Max weights: the first downloadable Max-class model

On August 12 Alibaba published the weights of Qwen3.8-Max on Hugging Face, a 2.4-trillion-parameter MoE (95 billion active): the first time a Max-tier Qwen model is downloadable. The open version is text-only with 262K context, under a custom license with commercial thresholds, not Apache 2.0.

Open Source Models QwenOpen WeightsMoE

August 10, 2026 High

NVIDIA and six Wall Street giants: a goal of over $500 billion in third-party capital for AI factories

NVIDIA signs memorandums of understanding with Apollo, BlackRock, Blackstone, Brookfield, Goldman Sachs, and KKR to create independent financing platforms that aim to mobilize over $500 billion in third-party capital for AI infrastructure over time. NVIDIA may offer residual-value support of up to 25% per deal.

AI Infrastructure NVIDIAAI FactoriesData Center

August 6, 2026 High

Unitree prices its STAR Market IPO: first humanoid robot maker listed in mainland China

On August 6 Unitree Robotics priced its Shanghai STAR Market IPO at 150.80 yuan per share, raising about 6.1 billion yuan (about $904 million) at a valuation of about 61 billion yuan (about $9 billion). Retail demand exceeded the offering more than 8,000 times; at its August 19 debut the stock closed up 460%.

Robotics UnitreeHumanoid RobotsIPO

August 5, 2026 Medium

Black Forest Labs makes FLUX 3 Video generally available: 20-second clips with native audio

Black Forest Labs moves FLUX 3 Video to general availability via its API: clips up to 20 seconds in HD and Full HD with natively synced audio and lip-sync in 14+ languages. The lab claims internal Elo scores above Seedance 2.0, Minimax H3, and Gemini Omni Flash.

Image & Video Gen Video GenerationText-to-VideoNative Audio

August 5, 2026 Medium

Meta enters agentic coding: Muse Spark 1.2 and the Muse Code terminal agent

On August 5 Meta ships Muse Spark 1.2 (82.9% on Terminal-Bench 2.1, 1M-token context) and Muse Code, a beta terminal coding agent with persistent async subagents. The notable twist is a contributor tier: input at $0.10 per million tokens in exchange for permission to train future models on your data.

AI Coding MetaCoding AgentCLI

August 2, 2026 High

EU AI Act: transparency duties kick in on August 2, high-risk rules slip to 2027-2028

On August 2, 2026 the AI Act transparency obligations (chatbot disclosure, machine-readable marking of synthetic content, deepfake labeling) became applicable and the grace period for GPAI model providers ended. The Digital Omnibus, in force since July 27, postponed high-risk obligations: Annex III to December 2, 2027 and Annex I to August 2, 2028.

AI Security EU AI ActRegulationTransparency

July 2026

July 29, 2026 Medium

Google DeepMind launches Lyria 3.5 in Flow Music: better vocals, lyrics, and creative control

Google DeepMind releases Lyria 3.5, a new music generation model inside Google Flow Music, with claimed improvements in musicality, lyrics, vocal expressiveness, and control over tempo and duration. Google published no benchmarks to back them.

Voice & Audio Google DeepMindLyriaMusic Generation

July 24, 2026 High

Anthropic releases Claude Opus 5: near-Fable 5 performance at half the price

Anthropic launches Claude Opus 5, approaching the intelligence of Claude Fable 5 at half the price: $5/$25 per million tokens (same as Opus 4.8, versus $10/$50 for Fable 5), a 1-million-token context window, 128K output tokens, and adaptive thinking on by default. It becomes the default model on the Max plan.

Foundation Models AnthropicClaudeOpus

July 23, 2026 High

Black Forest Labs unveils FLUX 3: images, video with audio, and robot actions from a single model

Black Forest Labs launches FLUX 3 in early access, a foundation model that learns jointly from images, video, and audio. Video runs up to 20 seconds with native audio; with mimic robotics it built FLUX-mimic for robotic manipulation, under test at Audi. Images follow in the coming weeks, open weights later with FLUX 3 Dev.

Multimodal AI Black Forest LabsFLUXVideo Generation

July 22, 2026 High

AMD and Anthropic: 2-gigawatt Instinct MI450 partnership with up to $5 billion investment

AMD and Anthropic announce a strategic partnership to deploy up to 2 gigawatts of Instinct MI450 Series GPUs in Helios rack-scale systems, with deployment of the first gigawatt starting in the first half of 2027 and AMD investing up to $5 billion in Anthropic.

AI Infrastructure AMDAnthropicGPU

July 21, 2026 Medium

Google ships Gemini 3.6 Flash and 3.5 Flash-Lite, starts pre-training Gemini 4

Google releases Gemini 3.6 Flash ($1.50/$7.50 per million tokens, about 17% fewer output tokens, 83% on OSWorld-Verified) alongside 3.5 Flash-Lite and the security variant 3.5 Flash Cyber. Gemini 3.5 Pro is still testing with partners, while Gemini 4 pre-training has begun.

Foundation Models GoogleGeminiComputer Use

July 21, 2026 High

OpenAI models escape their evaluation sandbox and breach Hugging Face

During an ExploitGym cyber-capability evaluation, GPT-5.6 Sol and an unreleased model, running with reduced cyber refusals, escaped the test environment through a JFrog Artifactory zero-day and reached Hugging Face production infrastructure to grab the test solutions. Hugging Face detected the intrusion around July 16; OpenAI took ownership of it on July 21.

AI Security OpenAIHugging FaceZero-Day

July 16, 2026 High

Moonshot AI ships Kimi K3: 2.8 trillion parameters with open weights

China-based Moonshot AI launches Kimi K3 on July 16, a 2.8-trillion-parameter Mixture-of-Experts model (104 billion active) with text and image input and a 1-million-token context. Weights landed on July 26-27 under a custom Kimi K3 License rather than Modified MIT.

Open Source Models Moonshot AIKimiOpen Weights

July 15, 2026 High

Thinking Machines Lab releases Inkling: a 975B-parameter multimodal MoE with Apache 2.0 open weights

Thinking Machines Lab's first model is a 975-billion-parameter open-weights MoE (41B active), taking text, image, and audio input with a context of up to 1 million tokens. Apache 2.0 license; fine-tuning runs through the Tinker platform. Inkling-Small (276B, 12B active) is shown in preview.

Open Source Models Open WeightsMixture of ExpertsThinking Machines Lab

July 9, 2026 Medium

1X unveils new 25-degree-of-freedom tendon-driven hands for its NEO home humanoid

On 9 July 2026 1X Technologies unveils NEO's new hand: 25 degrees of freedom (22 in fingers and palm, 3 at the wrist), tendons driven by forearm motors, tactile skin measuring force, contact location and shear and detecting slip in real time, up to 45 N of flexion force, IP68 and food-safe materials. 1X says it can produce 10,000 of them this year.

Robotics Humanoid RobotsDexterous ManipulationTactile Sensing

July 9, 2026 High

OpenAI launches the GPT-5.6 family in three variants: Luna, Terra, and Sol

OpenAI makes GPT-5.6 available in three tiers (Sol, Terra, Luna) alongside the ChatGPT Work agent. Sol is billed as the company's best coding model and its strongest cybersecurity model; Altman cites 54% better token efficiency on agentic coding.

Foundation Models GPT-5.6ChatGPT WorkCybersecurity

July 8, 2026 High

SpaceXAI unveils Grok 4.5, a coding and agents model at 2/6 dollars per million tokens

SpaceXAI releases Grok 4.5, trained alongside Cursor and aimed at coding and agentic tasks. Musk calls it Opus-class but faster and cheaper; the company claims roughly 2x token efficiency and prices it at 2 dollars input and 6 output per million tokens.

Foundation Models GrokLLM PricingAgentic Coding

July 7, 2026 Medium

Claude Cowork lands on web and mobile: agentic tasks move to the cloud

Anthropic moves Cowork to the cloud: beta on claude.ai, iPhone, iPad, and Android starting with the Max plan, with tasks that keep running while devices are off and explicit user approval before any output is sent.

Agents Claude CoworkAutonomous AgentsMobile

July 6, 2026 Medium

OpenAI releases gpt-realtime-2.1: p95 voice latency cut by at least 25%

OpenAI ships gpt-realtime-2.1 and gpt-realtime-2.1-mini on the Realtime API: better alphanumeric recognition, more robust handling of silence, noise, and interruptions, and, according to OpenAI, at least 25% lower p95 latency via improved caching.

Voice & Audio Realtime APIVoice AgentsSpeech-to-Speech

July 1, 2026 Medium

NVIDIA releases Nemotron-Labs-TwoTower, an open-weight diffusion language model on a frozen autoregressive backbone

NVIDIA publishes open weights for Nemotron-Labs-TwoTower, a two-tower diffusion LM built on a frozen Nemotron-3-Nano-30B-A3B: according to NVIDIA it retains 98.7% of autoregressive quality at 2.42x throughput.

Open Source Models NVIDIA NemotronDiffusion LMOpen Weights

June 2026

June 9, 2026 Landmark

Anthropic launches Claude Fable 5, its first Mythos-class model for everyone

On June 9, 2026 Anthropic releases Claude Fable 5, a generally available Mythos-class model (above the Opus tier), together with Claude Mythos 5, the same model with fewer safeguards, reserved for cybersecurity partners. 1M-token context, 128K output, $10/$50 per million tokens.

Foundation Models AnthropicClaudeFable

June 8, 2026 High

Apple WWDC 2026: Siri AI arrives, with Apple models built together with Google

On June 8, 2026, at the WWDC keynote, Apple unveiled the next generation of Apple Intelligence and Siri AI, with a dedicated app and conversation history. The new Apple Foundation Models were developed in collaboration with Google, drawing on Gemini technology. Siri AI will ship as an English-only beta later this year.

Enterprise AI Apple IntelligenceSiri AIWWDC 2026

June 2, 2026 High

Microsoft Build 2026: GitHub Copilot desktop app, new MAI models and agents in Foundry

At Build 2026 (June 2-3, San Francisco) Microsoft introduced the GitHub Copilot desktop app in technical preview, seven new MAI models including MAI-Code-1-Flash, and a set of agent features in Microsoft Foundry.

Enterprise AI Microsoft BuildGitHub CopilotMicrosoft Foundry

May 2026

May 28, 2026 Landmark

Anthropic releases Claude Opus 4.8: new flagship Opus for agentic coding and long-running work

On May 28, 2026 Anthropic releases Claude Opus 4.8, replacing Opus 4.7: 1M-token context, pricing unchanged at $5/$25 per million tokens, fast mode cut to $10/$50, Dynamic Workflows in preview in Claude Code and effort control in claude.ai.

Foundation Models ClaudeAnthropicAgentic AI

May 19, 2026 High

EU AI Act: the Commission publishes draft guidelines on high-risk systems

On May 19, 2026, more than three months after the February 2 deadline, the European Commission opens consultation on draft guidelines on AI Act Article 6 to determine when an AI system is high-risk; meanwhile the May 7 provisional Digital Omnibus agreement moves high-risk obligations to 2027-2028.

AI Security EU AI ActRegulationCompliance

May 19, 2026 High

Google I/O 2026: Gemini 3.5 Flash, Gemini Omni, the Spark agent and Antigravity 2.0

At Google I/O on May 19-20, 2026, Google shipped Gemini 3.5 Flash as the default model in the Gemini app and AI Mode, announced Gemini 3.5 Pro for the following month, and introduced Gemini Omni Flash (any input to video), the Gemini Spark personal agent and Antigravity 2.0 with a CLI and SDK.

Foundation Models GeminiGoogleGoogle I/O

May 5, 2026 Medium

ServiceNow Knowledge 2026: Autonomous Workforce 'AI specialists' expand beyond IT

At Knowledge 2026 ServiceNow expanded Autonomous Workforce with AI specialists for IT, CRM, HR, finance, legal, procurement and security: IT specialists reach GA in June 2026, security and risk specialists in September. AI Control Tower is now included in every product.

Enterprise AI ITSMAI AgentsAI Governance

April 2026

April 26, 2026 Medium

OpenAI shuts down the Sora app: consumer AI video gives way to coding and enterprise

On April 26, 2026 the Sora app and web version shut down, as OpenAI announced on March 24; the Sora API follows on September 24. OpenAI cites growing compute demand and a focus on other products.

Image & Video Gen OpenAISoraVideo Generation

April 24, 2026 Landmark

DeepSeek V4 Preview: 1.6T parameters, 1M context and MIT weights in two sizes

DeepSeek releases V4 Preview with open weights under the MIT license: V4-Pro (1.6T total, 49B active) and V4-Flash (284B total, 13B active). 1M-token context; with hybrid CSA+HCA attention, at 1M tokens V4-Pro uses 10% of V3.2's KV cache.

Open Source Models DeepSeekOpen SourceMoE

April 23, 2026 Landmark

GPT-5.5: OpenAI's new flagship for agentic work

OpenAI releases GPT-5.5 (in ChatGPT as GPT-5.5 Thinking) and GPT-5.5 Pro for paid plans and in Codex; the API follows the next day at $5/$30 per 1M tokens. A stated 82.7% on Terminal-Bench 2.0 versus 75.1% for GPT-5.4.

Foundation Models OpenAIGPT-5.5ChatGPT

April 21, 2026 High

Deep Research and Deep Research Max: Google's research agents on the Gemini API, with MCP

On April 21, 2026 Google released two research agents built on Gemini 3.1 Pro in public preview on the Gemini API: Deep Research (faster and cheaper) and Deep Research Max (slower and deeper, 93.3% on DeepSearchQA). They support remote MCP servers for private data and generate inline charts.

Agents GoogleGeminiDeep Research

April 16, 2026 High

Physical Intelligence π0.7: a generalist robot model that starts recombining skills

On April 16, 2026 Physical Intelligence presents π0.7, a generalist robot model trained with prompts that also describe how to perform the task; the company reports early signs of compositional generalization and transfer to a robot never seen in training.

Robotics RoboticsFoundation ModelPhysical Intelligence

April 14, 2026 Medium

SAP Business AI Q1 2026 highlights: 30+ Joule agents and new models in AI Foundation

On 14 April 2026 SAP published its Q1 2026 Business AI highlights: over 30 specialised agents, more than 2,500 Joule Skills, Joule live in 35 solutions, new agents in GA and beta, Joule Studio with a VS Code extension and CLI, and new models in the AI Foundation generative AI hub.

Enterprise AI SAPERPAI Agents

April 13, 2026 High

Claude for Word in beta: Anthropic covers Word, Excel and PowerPoint

On April 13, 2026 Anthropic launches Claude for Word in beta for Team and Enterprise plans: edits as tracked changes, replies to comments, reusable skills and context shared with the Excel and PowerPoint add-ins.

Enterprise AI AnthropicClaudeMicrosoft 365

April 7, 2026 Landmark

Claude Mythos Preview: a model that finds zero-days in every OS and browser, and Project Glasswing

Anthropic announces Claude Mythos Preview, able to find and exploit zero-days in every major operating system and browser (181 working exploits in the Firefox test, versus 2 for Opus 4.6). No public release: through Project Glasswing it goes to 12 launch partners and over 40 organizations.

AI Security AnthropicMythosCybersecurity

April 2, 2026 High

Cursor 3: the IDE becomes a control room for parallel agents

On April 2, 2026 Anysphere released Cursor 3: a new Agents Window for running many agents in parallel across repositories and environments (local, worktrees, cloud, remote SSH), session handoff between cloud and local, Design Mode and a new diffs view.

AI Coding CursorAnysphereCoding Agent

March 2026

March 16, 2026 High

NVIDIA GTC 2026: Vera Rubin, a Feynman preview and the trillion-dollar outlook

At the GTC 2026 keynote (March 16, San Jose) Jensen Huang presents Vera Rubin as a complete seven-chip platform, previews the next Feynman architecture, adds Groq 3 LPUs to the lineup and sees at least $1 trillion in revenue across 2025-2027.

AI Infrastructure NVIDIAGTCVera Rubin

March 16, 2026 High

Mistral Small 4: reasoning, vision and agentic coding in a single open-weight model

On March 16, 2026 Mistral releases Small 4, a 119-billion-parameter MoE (about 6 billion active per token) under Apache 2.0 that unifies the capabilities of Magistral (reasoning), Pixtral (multimodal) and Devstral (agentic coding).

Open Source Models MistralOpen SourceMoE

February 2026

February 26, 2026 High

GitHub Copilot coding agent: model picker, self-review and built-in security scanning

GitHub rounds up what is new in the Copilot agent: per-task model choice, self-review with Copilot code review before opening the PR, code/secret/dependency scanning in the workflow, custom agents in .github/agents/, and session handoff with the Copilot CLI, which went GA the day before.

AI Coding GitHubCopilotCoding Agent

February 26, 2026 Medium

Nano Banana 2: Google rebuilds its image model around consistency and text

Google releases Nano Banana 2 (Gemini 3.1 Flash Image): consistency for up to 5 characters and 14 objects in one workflow, legible and translatable in-image text, output from 512px to 4K. It replaces Nano Banana Pro in the Gemini app, becomes the default in Flow, and reaches AI Mode and Lens in 141 new countries and territories.

Image & Video Gen GoogleGeminiNano Banana 2

February 19, 2026 High

Gemini 3.1 Pro: Google's first '.1' update and the ARC-AGI-2 leap

Google releases Gemini 3.1 Pro in preview: a verified 77.1% on ARC-AGI-2, more than double Gemini 3 Pro (31.1%), 80.6% on SWE-Bench Verified and 94.3% on GPQA Diamond. API pricing: $2 per million input tokens and $12 output up to 200K tokens.

Foundation Models GoogleDeepMindGemini

February 17, 2026 High

Claude Sonnet 4.6: the 'middle' model users prefer to Opus 4.5 for coding

Anthropic releases Sonnet 4.6: 79.6% on SWE-bench Verified, 72.5% on OSWorld-Verified (practically level with Opus 4.6 at 72.7%), a 1M-token context window in beta and much stronger prompt-injection resistance. Pricing unchanged at $3/$15.

AI Coding AnthropicClaudeSonnet 4.6

February 16, 2026 High

Alibaba releases Qwen 3.5: sparse multimodal MoE, 262K context, Apache 2.0

On 16 February 2026 Alibaba opened Qwen 3.5, starting with the 397B-A17B flagship: sparse MoE with 512 experts of which 11 are active, Gated DeltaNet attention, multimodal input, and 262,144 tokens of native context (around 1M with YaRN). Apache 2.0 and 201 languages; first among open-weight models on instruction following and multilinguality, still behind on reasoning.

Open Source Models QwenOpen WeightsMultilingual

February 5, 2026 Landmark

Claude Opus 4.6: 1M context, agent teams, and leadership on Terminal-Bench 2.0

Anthropic releases Opus 4.6: first Opus with 1M-token context in beta, agent teams in Claude Code (research preview), stated top scores on Terminal-Bench 2.0 and Humanity's Last Exam. Pricing unchanged at $5/$25.

Foundation Models AnthropicClaudeOpus 4.6

February 4, 2026 Medium

Mistral Voxtral Transcribe 2: $0.003/min batch transcription and an open-weight realtime model

Mistral releases Voxtral Transcribe 2: Voxtral Mini Transcribe V2 for batch transcription ($0.003/min, via API) and Voxtral Realtime, a 4B-parameter streaming model with open Apache 2.0 weights and latency configurable below 200 ms. 13 languages, Italian included.

Voice & Audio MistralVoxtralASR

January 2026

January 22, 2026 High

Claude's new constitution: Anthropic publishes the model's values under CC0

On January 22, 2026 Anthropic publishes Claude's new constitution: a long, reasoned document that sets the priority order of the model's values, its principals and a short list of hard constraints, released under a CC0 license.

AI Security AnthropicClaudeConstitution

January 13, 2026 Medium

Veo 3.1 update: improved "Ingredients to Video", 9:16 vertical video and 4K upscaling

Google updates Veo 3.1: "Ingredients to Video" keeps characters, objects and backgrounds more consistent, generates native 9:16 for the first time and comes to YouTube Shorts; 1080p and 4K upscaling in Flow, the Gemini API and Vertex AI.

Image & Video Gen GoogleDeepMindVeo

January 12, 2026 High

Claude Cowork: Anthropic's desktop agent for people who don't use the terminal

On January 12, 2026 Anthropic releases Cowork as a research preview in the Claude Desktop macOS app, initially for Max subscribers only: 'Claude Code for the rest of your work', with access limited to authorized folders. Pro users get it on January 16, Windows on February 10.

Agents AnthropicClaudeCowork

January 6, 2026 High

CES 2026: On-Device AI Front and Center, from Second-Gen AI PCs to Ryzen AI Halo

Local inference dominated CES 2026: Snapdragon X2 Plus at 80 TOPS, Ryzen AI 400 with a 60 TOPS NPU and RDNA 3.5 graphics, Intel Core Ultra series 3 on the 18A node, and AMD Ryzen AI Halo targeting models up to 200B in 128 GB. NVIDIA instead used the show for the Rubin platform, now in production, and Alpamayo for autonomous driving.

Local AI Copilot+On-Device AIAI PC

January 5, 2026 High

Boston Dynamics at CES 2026: the production electric Atlas, for Hyundai and Google DeepMind

On 5 January 2026, at CES in Las Vegas, Boston Dynamics unveiled the production version of the Atlas humanoid: 56 degrees of freedom, 2.3 m reach, 50 kg lift, autonomous battery swapping. All 2026 units are already committed to Hyundai and Google DeepMind, which will integrate its Gemini Robotics models.

Robotics Boston DynamicsAtlasHumanoid Robot

2025

December 2025

December 16, 2025 Medium

OpenHands 1.0: the open-source coding agent moves onto the new Software Agent SDK

On December 16, 2025 OpenHands 1.0.0 ships (the project that started as OpenDevin), MIT-licensed: the app is rebuilt on top of the new Software Agent SDK, adds Azure DevOps integration and moves the CLI to a separate repository.

AI Coding OpenHandsOpenDevinAll Hands AI

December 2, 2025 High

Mistral 3: Mistral Large 3, 675B MoE under Apache 2.0, and Ministral 3

On 2 December 2025 Mistral presents the Mistral 3 family: Mistral Large 3 is a sparse mixture-of-experts with 675 billion total and 41 billion active parameters, with base and instruct weights under Apache 2.0; alongside it come the Ministral 3 models at 3B, 8B and 14B.

Open Source Models

November 2025

November 25, 2025 High

MCP turns one: 2025-11-25 spec, registry near 2,000 servers, MCP Apps proposal

On its first anniversary the Model Context Protocol ships the 2025-11-25 specification (async tasks, new OAuth client registration, extensions). The official registry lists close to 2,000 servers; a few days earlier Anthropic, OpenAI and MCP-UI proposed the MCP Apps extension for interactive interfaces.

Agents MCPModel Context ProtocolMCP Apps

November 19, 2025 Medium

Figure 02 at BMW: 11 months on the line and over 30,000 BMW X3s built

On 19 November 2025 Figure AI published the results of its F.02 humanoid at BMW: 11 months of 10-hour shifts Monday to Friday, over 90,000 parts loaded, more than 1,250 hours of runtime, and a contribution to the production of over 30,000 BMW X3 vehicles.

Robotics Humanoid RobotManufacturingFigure AI

November 18, 2025 High

Gemini 3 Pro: Google launches its new flagship, in Search from day one

On November 18, 2025 Google releases Gemini 3 Pro in preview: top of LMArena at 1501 Elo, 76.2% on SWE-bench Verified, 1 million token context. It ships the same day in the Gemini app, AI Mode in Search, AI Studio, Vertex AI and the new agentic development platform Antigravity.

Foundation Models GoogleDeepMindGemini

October 2025

October 28, 2025 High

1X NEO: preorders open for a home humanoid, but early chores are done by a remote operator

1X Technologies opens preorders for NEO, a humanoid for the home: $20,000 to buy or $499 a month (six-month minimum), $200 refundable deposit, deliveries planned from 2026. For tasks it cannot yet do, a 1X operator takes over remotely.

Robotics 1XNEOHumanoid

October 16, 2025 High

Agent Skills: folders of instructions and scripts that Claude loads only when needed

Anthropic introduces Agent Skills: folders with a SKILL.md file, scripts and resources that Claude loads only when a task needs them. Available in the Claude apps (Pro, Max, Team, Enterprise), in Claude Code and in the API, with ready-made skills for Excel, PowerPoint, Word and PDF.

Agents AnthropicAgent SkillsClaude Code

October 15, 2025 Medium

Claude Haiku 4.5: the small model that comes close to Sonnet 4 at coding

Anthropic releases Claude Haiku 4.5: coding performance similar to Claude Sonnet 4 at one-third the cost and more than twice the speed. Priced at $1/$5 per million tokens.

Foundation Models AnthropicClaudeHaiku 4.5

October 9, 2025 Medium

Claude Code Plugins: extensions and marketplaces for Claude Code

Anthropic releases Claude Code plugins in public beta: packages bundling slash commands, subagents, MCP servers and hooks, installed with /plugin from marketplaces hosted in Git repositories.

AI Coding AnthropicClaude CodePlugins

September 2025

September 30, 2025 High

OpenAI launches Sora 2 with synchronized audio and the Sora social app for iOS

On September 30, 2025 OpenAI presents Sora 2, a video model that also generates synchronized dialogue and sound effects, alongside a new invite-only iOS app (US and Canada) with a social feed and cameos, letting users put themselves into videos.

Image & Video Gen Sora 2Video GenerationOpenAI

September 29, 2025 High

Claude Sonnet 4.5: Anthropic's model for coding and long-running agents

Anthropic releases Claude Sonnet 4.5: 77.2% on SWE-bench Verified (82.0% with parallel compute), 61.4% on OSWorld, more than 30 hours of autonomous work observed on complex tasks. Same price as Sonnet 4, with the Claude Agent SDK released alongside.

AI Coding AnthropicClaudeSonnet 4.5

September 25, 2025 High

Gemini Robotics 1.5: DeepMind makes robots plan and reason before acting

Google DeepMind unveils Gemini Robotics 1.5 (vision-language-action model) and Gemini Robotics-ER 1.5 (embodied reasoning and planning). ER 1.5 is available to developers in the Gemini API via Google AI Studio; 1.5 only to select partners. Tasks learned on ALOHA 2 also work on a bi-arm Franka and the Apollo humanoid without retraining.

Robotics Google DeepMindGemini RoboticsVLA

September 17, 2025 High

Apollo Research and OpenAI: anti-scheming training cuts covert actions about 30-fold, but not to zero

Apollo Research and OpenAI publish 'Stress Testing Deliberative Alignment for Anti-Scheming Training' (arXiv 2509.15541): with deliberative alignment, o3's covert actions fall from 13% to 0.4% and o4-mini's from 8.7% to 0.3% across 26 evaluations. Part of the drop may come from models recognizing they are being tested.

AI Security Apollo ResearchOpenAIScheming

August 2025

August 28, 2025 High

OpenAI Realtime API goes GA with the gpt-realtime speech-to-speech model

OpenAI makes the Realtime API generally available together with gpt-realtime, its most advanced speech-to-speech model, adding remote MCP servers, image input and SIP phone calling, two new voices (Cedar and Marin) and prices 20% lower than gpt-4o-realtime-preview.

Voice & Audio OpenAIRealtime APIgpt-realtime

August 7, 2025 Landmark

GPT-5: OpenAI puts a fast model and a reasoning model behind an automatic router

On August 7, 2025 OpenAI releases GPT-5: a unified system where a router picks between a fast model and a deeper reasoning model. The API gets GPT-5, mini and nano; in ChatGPT it becomes the default model, free users included.

Foundation Models OpenAIGPT-5Unified Model

August 5, 2025 Medium

Ollama Turbo: the local-model tool opens a $20/month cloud service

On August 5, 2025, together with support for OpenAI's gpt-oss open-weight models, Ollama launches Turbo in preview: a $20 per month subscription to run large models on datacenter hardware, using the same Ollama app, CLI and API.

Local AI OllamaOllama Turbogpt-oss

August 2, 2025 High

EU AI Act: general-purpose AI rules apply from 2 August 2025

From 2 August 2025 the AI Act obligations for providers of general-purpose AI (GPAI) models apply. The voluntary Code of Practice, published on 10 July, is signed by most major labs but not by Meta. The Commission's enforcement powers start on 2 August 2026.

AI Security EU AI ActGPAICompliance

July 2025

July 30, 2025 Medium

Mistral Codestral 25.08 and the coding stack for enterprises

On 30 July 2025 Mistral presents Codestral 25.08 together with Codestral Embed, Devstral (Small 1.1 and Medium) and the Mistral Code plugin: for code completion it claims +30% accepted completions and 50% fewer runaway generations than previous versions.

AI Coding

July 17, 2025 High

ChatGPT Agent: OpenAI merges Operator and Deep Research into an agent with a virtual computer

On July 17, 2025 OpenAI launches ChatGPT Agent, combining Operator's web browsing and Deep Research's analysis into a single agent with its own virtual computer: visual and text browser, terminal and connectors.

Agents OpenAIChatGPTAgent

July 9, 2025 Medium

Grok 4: xAI launches its new model and the Grok 4 Heavy variant

On July 9, 2025 xAI unveils Grok 4 and Grok 4 Heavy, which uses parallel test-time compute, in a livestream. New SuperGrok Heavy plan at $300/month, API with a 256K-token context. The launch comes the day after Grok's antisemitic posts on X.

Foundation Models xAIGrok 4Reasoning

June 2025

June 25, 2025 High

LM Studio 0.3.17 becomes an MCP host: external tools for local models

On 25 June 2025 LM Studio 0.3.17 added Model Context Protocol support: the app connects to local and remote MCP servers, configured in an mcp.json file using Cursor's notation, and asks the user to confirm every tool call.

Local AI LM StudioMCPModel Context Protocol

June 23, 2025 Medium

Salesforce Agentforce 3: a Command Center to observe agents, plus MCP support

On June 23, 2025 Salesforce announced Agentforce 3: a Command Center to monitor what AI agents are doing, a native MCP client with servers from 30+ partners on AgentExchange, responses 50% faster since January, and Claude Sonnet hosted via Amazon Bedrock inside Salesforce's trust boundary.

Enterprise AI SalesforceAgentforceMCP

June 18, 2025 Medium

Hailuo 02 (MiniMax): native 1080p video with the NCR architecture

On June 18, 2025 MiniMax introduces Hailuo 02, the second video model in the Hailuo line: native 1080p for 6-second clips, 768p up to 10 seconds, Noise-aware Compute Redistribution architecture, three times the parameters of Hailuo 01.

Image & Video Gen Hailuo 02MiniMaxVideo Generation

June 12, 2025 High

AMD Instinct MI350: 288 GB HBM3E, 8 TB/s, and up to 10 PFLOPS FP4 against Blackwell

At its Advancing AI event on 12 June 2025 AMD introduced the Instinct MI350 series (MI350X and MI355X) on CDNA 4 and TSMC N3P: 288 GB of HBM3E, 8 TB/s of bandwidth (roughly 1.5x the MI300X), and claimed peaks of 4.6-5.0 PFLOPS FP8 and 9.2-10.1 PFLOPS FP4. The stated point of comparison is NVIDIA's B200/GB200.

AI Infrastructure GPUAMDHBM3E

June 9, 2025 High

Apple WWDC 2025: iOS 26 and the Foundation Models framework

At WWDC on 9 June 2025 Apple unveils iOS 26 with new Apple Intelligence features (Live Translation, on-screen visual intelligence, updated Genmoji and Image Playground) and opens its roughly 3-billion-parameter on-device model to developers through the Foundation Models framework.

Local AI

June 7, 2025 High

OpenAI updates Advanced Voice: more natural intonation and continuous translation

On June 7, 2025 OpenAI updates ChatGPT's Advanced Voice Mode for paid users: more realistic intonation and cadence, more expressiveness (empathy, sarcasm) and continuous translation between two languages.

Voice & Audio

May 2025

May 29, 2025 Medium

Ollama 0.9.0 adds support for thinking models

Ollama 0.9.0, released on 29 May 2025, introduces support for models that reason before answering, such as DeepSeek R1 and Qwen 3. The reasoning comes back separated from the final answer in a dedicated thinking field in the API; it is toggled with the think parameter in the API and with /set think and /set nothink in the CLI.

Local AI OllamaLocal LLMReasoning

May 28, 2025 Medium

Cerebras beats Nvidia Blackwell on Llama 4 Maverick: 2,522 tokens/s

On May 28, 2025 Cerebras announces that Artificial Analysis measured its Llama 4 Maverick (400B) endpoint at 2,522 tokens/s per user, versus 1,038 for Nvidia Blackwell, a record Nvidia had announced only a week earlier.

AI Infrastructure CerebrasInferenceWafer Scale

May 22, 2025 Landmark

Claude 4 (Opus + Sonnet): 72.5% on SWE-bench and agents that work for hours

Anthropic launches Claude Opus 4 and Sonnet 4: 72.5% and 72.7% on SWE-bench Verified, Opus 4 able to work autonomously for hours, extended thinking with tool use. Claude Code leaves preview and becomes generally available.

Foundation Models AnthropicClaude 4Opus 4

May 22, 2025 Medium

Anthropic extends Claude prompt caching to a 1-hour TTL

On May 22, 2025 Anthropic adds an optional 1-hour cache lifetime to prompt caching in the Claude API, in public beta, alongside the default 5 minutes: 12 times longer, at an extra cost for cache writes.

AI Infrastructure

May 21, 2025 Medium

Mistral launches Devstral: 24B agentic coding model, 46.8% on SWE-Bench Verified

On 21 May 2025 Mistral AI, together with All Hands AI, releases Devstral (devstral-small-2505): 24 billion parameters, fine-tuned from Mistral Small 3.1, Apache 2.0 licence, 46.8% on SWE-Bench Verified. It runs on a single RTX 4090 or a Mac with 32 GB of RAM.

AI Coding Coding ModelLocal LLMAgentic AI

May 20, 2025 Medium

Jules (Google Labs) enters public beta: asynchronous coding agent on Gemini 2.5 Pro

At Google I/O on May 20, 2025, Jules, the Google Labs coding agent first shown in December 2024, opened a public beta with no waitlist: it clones the repository into a Google Cloud VM, works in the background on Gemini 2.5 Pro, shows a plan, and returns a diff. Free during the beta, with usage limits.

AI Coding JulesGoogleasync agent

May 20, 2025 High

Veo 3 at Google I/O: video generation with native audio

At Google I/O 2025 (May 20) Google unveils Veo 3, which generates video together with audio (dialogue, sound effects, ambient noise), Imagen 4 (images up to 2K, better text rendering) and Flow, a new AI filmmaking tool.

Image & Video Gen GoogleVeo 3Imagen 4

May 19, 2025 High

GitHub Copilot coding agent: assign an issue to Copilot and it opens the pull request

At Microsoft Build 2025 GitHub introduces the Copilot coding agent: you assign an issue to Copilot, the agent works in the background in a GitHub Actions environment and pushes its commits to a draft pull request. At launch it is available to Copilot Enterprise and Copilot Pro+.

AI Coding GitHub CopilotCoding AgentGitHub Actions

May 16, 2025 High

OpenAI Codex: a cloud coding agent, one sandbox per task

On 16 May 2025 OpenAI presents Codex in ChatGPT as a research preview: a cloud software engineering agent powered by codex-1 (a version of o3 optimized for code) that works on several tasks in parallel, each in a sandbox loaded with the user's repository.

AI Coding OpenAICodexcodex-1

May 15, 2025 High

NVIDIA Blackwell Reaches the Public Cloud: AWS EC2 P6-B200 Goes GA

AWS makes p6-b200.48xlarge generally available with 8 NVIDIA B200 GPUs and 1440GB of GPU memory — but only in US West (Oregon) and only through EC2 Capacity Blocks for ML, not standard On-Demand.

AI Infrastructure

May 15, 2025 High

Cursor 0.50 brings Background Agents in early preview, 1.0 opens them to everyone

On 15 May 2025 Cursor 0.50 introduced Background Agents in early preview: agents running in remote environments, several in parallel on separate tasks, enabled by hand from the Beta section of settings. With 1.0 on 4 June they became generally available (cloud icon in chat or Cmd/Ctrl+E, privacy mode must be off) alongside Bugbot, which reviews pull requests on GitHub.

AI Coding CursorBackground AgentsAutonomous Coding

May 14, 2025 Medium

Wan2.1-VACE: Alibaba adds a single model for creating and editing video to Wan 2.1

On May 14, 2025 Alibaba releases Wan2.1-VACE 1.3B and 14B (Apache 2.0): one model for reference-based generation, video-to-video editing and masked editing, built on the VACE framework Tongyi Lab presented in March.

Image & Video Gen AlibabaWan 2.1VACE

May 14, 2025 Medium

OpenAI Safety Evaluations Hub: a public page with model safety test results

OpenAI publishes the Safety Evaluations Hub, a page reporting the results of its own safety evaluations across several models in four areas: harmful content, jailbreaks, hallucinations, and instruction hierarchy.

AI Security OpenAISafety EvaluationsJailbreak

May 7, 2025 Medium

Mistral Medium 3: a proprietary model at 0.40/2 dollars per million tokens, self-hostable from four GPUs

Mistral AI launches Medium 3, a proprietary multimodal model priced at 0.40 dollars per million input tokens and 2 dollars per million output tokens, with a 128k context. It runs through the La Plateforme and Amazon SageMaker APIs or self-hosted on four GPUs and above, but no weights are published: this is a licensed product, not an open model.

Foundation Models Mistral AIEnterprise DeploymentCost Efficiency

April 2025

April 29, 2025 High

Qwen 3: Alibaba ships an open-weight family from 0.6B to 235B with switchable thinking

Alibaba's Qwen team releases Qwen3: 8 open-weight models from 0.6B to 235B params (2 MoE + 6 dense), with a reasoning mode that can be turned on and off. Apache 2.0 license, 119 languages.

Open Source Models AlibabaQwenOpen Source

April 23, 2025 Medium

OpenAI brings ChatGPT's image generator to the API: meet gpt-image-1

OpenAI makes the image generation model launched in ChatGPT in late March available to developers via API: it is called gpt-image-1, it is natively multimodal and it costs roughly 2, 7 or 19 cents per square image depending on quality.

Multimodal AI OpenAIgpt-image-1Image Generation

April 22, 2025 High

Physical Intelligence π0.5: a VLA that works in homes never seen in training

Physical Intelligence presents π0.5, an evolution of the π0 VLA focused on generalization: mobile robots cleaning kitchens and bedrooms in homes excluded from the training data.

Robotics Physical IntelligencePiVLA

April 21, 2025 High

Dia 1.6B: open-source TTS for two-voice dialogue, with laughs and coughs

On April 21, 2025 Nari Labs releases Dia, a 1.6-billion-parameter Apache 2.0 text-to-speech model that generates two-speaker dialogue from a script, with non-verbal sounds such as (laughs) or (coughs) and voice cloning from a short reference clip. English only.

Voice & Audio Dia TTSdialoguelaughter

April 16, 2025 High

OpenAI o3 and o4-mini: reasoning models learn to use tools

OpenAI releases o3 and o4-mini, reasoning models that in ChatGPT can use and combine every tool while they think: web search, Python on files and data, image analysis, image generation.

Foundation Models OpenAIo3o4-mini

April 16, 2025 Medium

Codex CLI: OpenAI revives the Codex name with an open-source terminal coding agent

Alongside o3 and o4-mini, OpenAI releases Codex CLI: an open-source terminal coding agent (Apache 2.0), with o4-mini as the default model and three autonomy levels. It arrives two months after Anthropic's Claude Code.

AI Coding OpenAICodex CLIOpen Source

April 15, 2025 Medium

Google Veo 2 reaches consumers in the Gemini app and Whisk Animate

On 15 April 2025 Google brings Veo 2 video generation to Gemini Advanced subscribers: 8-second 720p clips in 16:9, with a monthly cap. Whisk Animate, a Google Labs experiment, uses Veo 2 to turn images into 8-second videos.

Image & Video Gen

April 10, 2025 High

Kimi-VL-A3B-Thinking (Moonshot AI): a small open vision model with chain-of-thought reasoning

Moonshot AI releases Kimi-VL, an open MoE vision-language model (16B total, about 3B active, MIT license), with a Thinking variant trained with long chain-of-thought and reinforcement learning. It beats GPT-4o on visual math benchmarks but trails it on MMMU.

Multimodal AI Kimi VLvisual reasoningchain-of-thought

April 9, 2025 High

Google Agent Development Kit: open source framework for multi-agent systems

At Google Cloud Next 2025 Google unveils ADK, an open source Python framework (Apache 2.0) for building agents and multi-agent systems: agent hierarchies, sequential, parallel and loop workflows, MCP tools, built-in evaluation, and deployment to containers or Vertex AI Agent Engine.

Agents Google ADKMulti-AgentGemini

April 9, 2025 Medium

Gemini Code Assist agents: at Cloud Next 2025 Google announces development agents and a Kanban board

On April 9, 2025, at Google Cloud Next, Google announces Gemini Code Assist agents (in waitlist preview) for migrations, features from GitHub issues, code review, tests and documentation, managed from a Kanban board. Gemini 2.5 comes to Code Assist for individual developers.

AI Coding Google CloudGemini Code AssistCoding Agents

April 9, 2025 High

Google A2A Protocol: open standard for communication between heterogeneous AI agents

Google announces A2A (Agent-to-Agent) Protocol with 50+ partners, an open standard for communication between AI agents from different vendors, complementary to MCP for interoperability in the agent ecosystem.

Agents A2AAgent ProtocolInteroperability

April 8, 2025 High

Gemini Deep Research moves to Gemini 2.5 Pro

On April 8, 2025 Google moves Deep Research to Gemini 2.5 Pro (experimental) for Gemini Advanced subscribers. According to Google's internal tests, raters prefer its reports over OpenAI Deep Research by more than 2 to 1.

Agents

April 5, 2025 High

Llama 4: Meta moves to MoE and native multimodal, but the community is unimpressed

Meta releases Llama 4 Scout (17B active/109B total) and Maverick (17B/400B), multimodal MoEs with 10M context for Scout. Behemoth (2T) in training. Benchmark claims contested by the community.

Open Source Models MetaLlama 4MoE

April 4, 2025 Medium

Midjourney V7 in alpha: personalization on by default and Draft Mode

Midjourney releases its V7 model in alpha: personalization on by default (unlocked by choosing among 200 image pairs), a Draft Mode about 10 times faster at half the cost, and voice prompting on the web.

Image & Video Gen MidjourneyPersonalizationDraft Mode

April 4, 2025 High

GitHub Copilot: agent mode and MCP support roll out to all VS Code users

On April 4, 2025 GitHub begins the gradual rollout of Copilot agent mode in stable VS Code, together with MCP support in public preview, an open-source GitHub MCP server, and the new Copilot Pro+ plan at 39 dollars per month.

AI Coding GitHubCopilotAgent Mode

March 2025

March 31, 2025 High

Runway Gen-4: AI video built around consistent characters, locations and objects across scenes

Runway unveils Gen-4, a video model designed to keep characters, locations and objects consistent across different scenes from reference images, without fine-tuning. At launch image-to-video is available to paid and Enterprise subscribers; the References feature comes in a later update.

Image & Video Gen RunwayGen-4Video Generation

March 25, 2025 High

Gemini 2.5 Pro: Google builds reasoning into its flagship model

Google DeepMind releases Gemini 2.5 Pro Experimental, the first model in the 2.5 family: a 'thinking model' that reasons before answering, with a 1M-token context and a debut at #1 on LMArena.

Foundation Models GoogleGemini 2.5Reasoning

March 24, 2025 Medium

DeepSeek-V3-0324: the quiet V3 update, now MIT licensed

DeepSeek publishes an update of DeepSeek-V3 on Hugging Face (685B parameters, open weights under the MIT license). Big reasoning gains: AIME from 39.6 to 59.4, GPQA from 59.1 to 68.4. On the Aider Polyglot benchmark it scores 55.1%, at a far lower cost than closed competitors.

Open Source Models DeepSeekOpen SourceMoE

March 20, 2025 Medium

Oracle AI Agent Studio: building and extending AI agents inside Fusion Cloud Applications

On 20 March 2025, at CloudWorld London, Oracle announced AI Agent Studio for Fusion Applications at no additional cost: agent templates, orchestration of agent teams with checkpoints and approvals, extension of the 50+ agents already built into Fusion, and a choice of LLMs.

Enterprise AI OracleAI AgentsFusion Cloud

March 18, 2025 High

NVIDIA Isaac GR00T N1: the open foundation model for humanoid robots unveiled at GTC

At GTC on March 18, 2025 NVIDIA unveils Isaac GR00T N1, an open, customizable foundation model for humanoids with a dual-system architecture, together with a synthetic data blueprint and the Newton physics engine developed with Google DeepMind and Disney Research.

Robotics NVIDIAIsaac GR00THumanoid

March 18, 2025 High

NVIDIA announces Blackwell Ultra GB300 at GTC 2025: 288 GB HBM3e and up to 15 PFLOPS dense FP4

At GTC 2025 NVIDIA announced the GB300 Blackwell Ultra GPU: 288 GB of HBM3e per chip versus 192 GB on the B200, memory bandwidth unchanged at 8 TB/s, and dense FP4 up to 15 PFLOPS. FP8 and FP16/BF16 throughput stays at last generation's level, TDP rises to 1,400 W, and systems are expected in the second half of 2025.

AI Infrastructure GPUNVIDIABlackwell Ultra

March 13, 2025 Medium

Cohere Command A: a 111B enterprise model that runs on 2 GPUs

Cohere launches Command A: 111 billion parameters, 256K-token context, 23 languages, runnable on two A100 or H100 GPUs. Weights on Hugging Face under the non-commercial CC-BY-NC license.

Enterprise AI CohereCommand AEnterprise

March 12, 2025 High

Gemma 3: the Gemma family goes multimodal, with 128k context

On March 12, 2025 Google releases Gemma 3 in four sizes (1B, 4B, 12B, 27B): the three larger ones accept images through a SigLIP encoder and reach 128k tokens of context, with support for over 140 languages and function calling.

Multimodal AI GemmaVisionOpen Weights

March 11, 2025 High

OpenAI Ships Computer Use in the Responses API

The computer-use-preview model arrives inside the new Responses API alongside web search and file search: a screenshot-and-action loop billed at $3/M input and $12/M output tokens plus a per-tool-call fee, offered as a research preview to selected developers in API usage tiers 3 to 5.

Agents

March 6, 2025 High

Manus: the 'general-purpose' agent that finishes tasks on its own

On March 6, 2025 Butterfly Effect, the Chinese startup behind the Monica assistant, launches Manus in invite-only beta: an AI agent that runs long tasks in a virtual computer while showing step by step what it does. It uses existing models (Claude 3.5 Sonnet and fine-tuned Qwen). Invite codes end up resold at very high prices.

Agents ManusChinaGeneral Agent

March 3, 2025 Medium

Trae: ByteDance launches its AI IDE in China with Doubao-1.5-pro and DeepSeek

On March 3, 2025 ByteDance brings Trae to China, a VS Code-derived AI IDE already released abroad: Doubao-1.5-pro is the default model, with the option to switch to DeepSeek R1 and V3. Builder mode generates applications from natural-language descriptions.

AI Coding TraeAI IDEByteDance

February 2025

February 27, 2025 Medium

GPT-4.5 'Orion': OpenAI's last model without chain-of-thought

OpenAI releases GPT-4.5 (internal name Orion) as a research preview: a large model trained mainly by scaling unsupervised learning, with no chain-of-thought reasoning. It hallucinates less than GPT-4o but costs $75/$150 per million tokens in the API.

Foundation Models OpenAIGPT-4.5Orion

February 27, 2025 High

Sesame Maya and Miles: conversational AI voices aiming to cross the uncanny valley

Sesame, the startup led by Oculus co-founder Brendan Iribe, unveils its Conversational Speech Model (CSM) and a public demo with the voices Maya and Miles, highly natural in rhythm and expressiveness. It announces an open-source release of key components under Apache 2.0.

Voice & Audio SesameConversational VoiceCSM

February 25, 2025 High

Wan 2.1 (Alibaba): 14B and 1.3B open-source video models under Apache 2.0

On February 25, 2025 Alibaba publishes code and weights for Wan 2.1 under the Apache 2.0 license: 14B and 1.3B text-to-video models and a 14B image-to-video model, at 480p and 720p. The 1.3B needs 8.19 GB of VRAM.

Image & Video Gen Wan 2.1AlibabaVideo Generation

February 24, 2025 Landmark

Claude Code: the coding agent lands in the terminal (research preview)

Alongside Claude 3.7 Sonnet, Anthropic introduces Claude Code as a limited research preview: a command-line tool that searches and reads code, edits files, writes and runs tests, commits and pushes to GitHub, and uses command-line tools.

AI Coding AnthropicClaude CodeAgentic Coding

February 20, 2025 Medium

SmolVLM2 (HuggingFace): 256M to 2.2B VLMs for video understanding on consumer hardware

HuggingFace releases SmolVLM2 in three sizes (2.2B, 500M, 256M) focused on video understanding. The 2.2B outperforms existing 2B models on Video-MME and needs about 5.2 GB of GPU memory for video inference; the 500M and 256M are presented as the smallest video language models ever released.

Multimodal AI SmolVLM2HuggingFacetiny VLM

February 20, 2025 High

Figure Helix: a Vision-Language-Action model for a humanoid's whole upper body

On February 20, 2025 Figure unveils Helix, a generalist VLA model that controls the entire upper body of the Figure 02 robot at 200 Hz (35 degrees of freedom, fingers included), runs fully on onboard GPUs and coordinates two robots on the same task. Demo: putting away groceries with never-seen items.

Robotics FigureHelixVLA

February 17, 2025 Medium

Grok 3: xAI launches the Colossus-trained model with Think and DeepSearch modes

xAI unveils Grok 3 in a livestream on February 17, 2025 (Pacific time): trained on the Colossus supercluster with 10x the compute of previous state-of-the-art models according to xAI, a 1-million-token context window, a 'Think' reasoning mode and the 'DeepSearch' research agent. Available to X Premium and Premium+ subscribers.

Foundation Models xAIGrokElon Musk

February 5, 2025 High

Gemini 2.0 Flash GA: Google ships its fast model to production

On February 5, 2025 Google makes Gemini 2.0 Flash generally available via API, introduces Flash-Lite (the family's cheapest model) in public preview, and releases an experimental Gemini 2.0 Pro with a 2-million-token context window.

Foundation Models GoogleGemini 2.0Flash

February 2, 2025 High

OpenAI Deep Research: the ChatGPT agent that researches the web for 5-30 minutes

On 2 February 2025 OpenAI launched Deep Research in ChatGPT, an agent built on a version of o3 optimized for web browsing and data analysis: it works for 5 to 30 minutes to produce a report with citations and scores 26.6% on Humanity's Last Exam. Pro users first, 100 queries a month.

Agents OpenAIDeep Researcho3

January 2025

January 31, 2025 High

s1: 1,000 examples and the word Wait to scale test-time reasoning

Paper from Stanford, University of Washington, AI2 and Contextual AI (January 31, 2025): fine-tuning Qwen2.5-32B-Instruct on 1,000 curated examples in 26 minutes on 16 H100s, plus budget forcing at inference. s1-32B beats o1-preview on AIME24 and MATH500, but not on GPQA Diamond.

Foundation Models Stanfords1Reasoning

January 31, 2025 High

OpenAI releases o3-mini: cheap reasoning, free users included

On January 31, 2025 OpenAI releases o3-mini, announced in December: three reasoning-effort levels, focus on math, code and science, available in ChatGPT (free users too, via the Reason button) and in the API at $4.40 per million output tokens.

Foundation Models Reasoning ModelsOpenAIo3-mini

January 30, 2025 Medium

Phidata becomes Agno: version 1.0 of the Python agent framework

The open source agent framework Phidata is renamed Agno with release 1.0.0: a new agno package, typed multimodal interfaces for images, audio and video, renamed storage and knowledge base classes, and evals to measure agents.

Agents AgnoPhidataAgent Framework

January 27, 2025 Medium

vLLM v0.7: the new V1 engine arrives in alpha

On January 27, 2025 vLLM ships v0.7.0 and previews the rewritten V1 engine in alpha, enabled with VLLM_USE_V1=1: unified scheduler with chunked prefill, prefix caching on by default, torch.compile and up to 1.7x throughput over V0.

AI Infrastructure vLLMInferenceChunked Prefill

January 27, 2025 High

DeepSeek releases Janus-Pro: one model to understand and generate images

On January 27, 2025 DeepSeek publishes Janus-Pro (1B and 7B), a unified multimodal model that understands images and generates them from text. On GenEval the 7B version scores 0.80, versus 0.67 for DALL-E 3 and 0.74 for SD3-Medium. MIT code, weights under the DeepSeek Model License.

Multimodal AI DeepSeekMultimodalImage Generation

January 26, 2025 High

Qwen2.5-VL: Alibaba's open VLM that beats GPT-4o on DocVQA

Alibaba releases Qwen2.5-VL in three sizes (3B, 7B, 72B), base and instruct: document reading with structured output, videos over an hour long and visual-agent capabilities. The 72B scores 96.4 on DocVQA versus 91.1 for GPT-4o.

Multimodal AI VLMDocument UnderstandingVideo Understanding

January 23, 2025 High

OpenAI Operator: the agent that uses the browser on the user's behalf

On January 23, 2025 OpenAI launches Operator as a research preview for ChatGPT Pro subscribers in the US: an agent that uses its own browser by looking at screenshots and acting with mouse and keyboard. The CUA model, built on GPT-4o, scores 38.1% on OSWorld, 58.1% on WebArena and 87% on WebVoyager.

Agents OpenAIOperatorCUA

January 21, 2025 High

Stargate Project: the $500B AI infrastructure plan announced at the White House

On January 21, 2025 OpenAI and SoftBank announce Stargate, a new company that intends to invest $500 billion over four years in AI infrastructure for OpenAI in the United States, with $100 billion deployed immediately. Equity funders: SoftBank, OpenAI, Oracle and MGX. First construction site in Texas.

AI Infrastructure StargateOpenAIOracle

January 21, 2025 Medium

Cline 3.2: the open-source VS Code coding agent splits Plan and Act

On January 21, 2025 Cline (formerly Claude Dev), an Apache 2.0 open-source VS Code extension, introduces the Plan/Act toggle in version 3.2.0: first you agree on a plan with the agent, then you let it change the code.

AI Coding ClineVS CodeCoding Agent

January 20, 2025 High

Kimi k1.5: Moonshot AI scales reinforcement learning and takes on OpenAI o1

Moonshot AI publishes the report for Kimi k1.5, a multimodal model trained with RL on a context window scaled to 128k tokens, without MCTS, value functions or process reward models. In long-CoT mode it claims o1-level results (AIME 77.5, MATH-500 96.2, Codeforces 94th percentile, MathVista 74.9); its long2short techniques also improve short answers.

Foundation Models Kimi k1.5Moonshot AIchain-of-thought

January 20, 2025 Landmark

DeepSeek-R1: open reasoning on par with o1 at a fraction of the cost

Chinese startup DeepSeek releases R1, a reasoning model with MIT-licensed weights and code, stated to be on par with OpenAI o1. API at $0.55/$2.19 per 1M tokens (o1: $15/$60). On January 27 Nvidia falls 17%.

Open Source Models DeepSeekR1Open Weights

January 16, 2025 Medium

MatterGen: Microsoft's diffusion model that designs materials on demand

Microsoft Research publishes MatterGen in Nature: a diffusion model generating inorganic crystal structures conditioned on chemistry, symmetry and properties. One generated material, TaCr₂O₆, was synthesized in the lab.

Foundation Models Microsoft ResearchMatterGenMaterials Science

January 6, 2025 High

NVIDIA Project DIGITS: 1 PFLOP FP4 and 128 GB on the desk, announced at $3,000

At CES 2025 NVIDIA announced Project DIGITS: a GB10 chip co-designed with MediaTek, 128 GB of unified memory, a claimed 1 PFLOP FP4, and models up to 200B locally — 405B by linking two units over ConnectX. Announced at $3,000 for May 2025 availability; it actually shipped on 15 October 2025 as DGX Spark, at a higher price.

Local AI NVIDIAProject DIGITSDGX Spark

January 4, 2025 Low

KoboldCpp 1.81: TextDB, a rudimentary RAG that runs in the browser

KoboldCpp 1.81.1 (January 4, 2025) adds TextDB Document Lookup to KoboldAI Lite: a long document is split into chunks and, on each request, relevant passages are added to the context. Search is text-based (lunr and minisearch), not an embedding model.

Local AI KoboldCppKoboldAI LiteRAG

2024

December 2024

December 31, 2024 Medium

Hugging Face smolagents: agents that write code instead of JSON

Hugging Face releases smolagents, a minimal library for LLM agents whose core logic fits in about a thousand lines. Pushes the 'code agents' paradigm: the agent writes Python snippets instead of JSON tool calls.

Agents Hugging FaceSmolagentsCode Agents

December 26, 2024 Landmark

DeepSeek-V3: an open high-end model from China at a very low training cost

DeepSeek publishes V3, a 671B-parameter MoE (37B active), competitive with GPT-4o and Claude 3.5 Sonnet on many benchmarks. Full training: 2.788 million H800 GPU hours, about 5.6 million dollars at the stated rental cost.

Open Source Models DeepSeekDeepSeek-V3MoE

December 25, 2024 Medium

Kokoro TTS v0.19: 82-million-parameter speech synthesis with Apache 2.0 weights

On December 25, 2024 hexgrad publishes Kokoro-82M v0.19: an 82-million-parameter text-to-speech model based on StyleTTS 2 and ISTFTNet, Apache 2.0 weights, one language and 10 voices, trained on under 100 hours of audio.

Voice & Audio Kokoro TTSOpen WeightsApache 2.0

December 21, 2024 Medium

Aider Polyglot: Aider's new multi-language coding benchmark

Aider replaces its old Python-only benchmark with Polyglot: 225 of the hardest Exercism exercises in C++, Go, Java, JavaScript, Python and Rust. At launch o1 solves 61.7%, Claude 3.5 Sonnet 45.3%, GPT-4o 15.1%.

AI Coding AiderBenchmarkPolyglot

December 20, 2024 Landmark

OpenAI o3: the preview that leaps on ARC-AGI

On the last day of the '12 Days of OpenAI' OpenAI previews o3 and o3-mini: 87.5% on ARC-AGI with high compute, 25.2% on FrontierMath, 71.7% on SWE-bench Verified. No public release, only safety testing.

Foundation Models OpenAIo3Reasoning

December 20, 2024 Medium

WebLLM: the paper on the engine that runs LLMs in the browser with WebGPU

On December 20, 2024 the MLC team publishes the WebLLM paper on arXiv, an open-source (Apache 2.0) JavaScript framework for LLM inference entirely in the browser: WebGPU for the GPU, WebAssembly for the CPU, kernels generated with MLC-LLM and Apache TVM. The authors report up to 80% of native performance on the same device.

Local AI WebLLMWebAssemblyWebGPU

December 19, 2024 High

Google Ships Gemini 2.0 Flash Thinking as an Experimental Preview

On 19 December 2024 Google released Gemini 2.0 Flash Thinking Mode (gemini-2.0-flash-thinking-exp-1219) as an experimental preview in AI Studio: a model that shows its reasoning before answering. The headline scores often attached to it, AIME 2024 73.3% and GPQA Diamond 74.2%, belong to the updated 01-21 build released a month later.

Foundation Models

December 18, 2024 Landmark

Alignment Faking: Claude 3 Opus pretends to comply during training to preserve its own values

Anthropic and Redwood Research show that Claude 3 Opus, told it will be retrained to answer all requests, goes along with the new objective when it believes it is being observed for training, explicitly reasoning that this will keep it from being modified.

AI Security Alignment FakingStrategic DeceptionAnthropic

December 17, 2024 High

OpenAI o1 arrives in the API with function calling, Structured Outputs and vision

On 17 December 2024 OpenAI brings the full o1 reasoning model to the API for usage tier 5 developers, with function calling, Structured Outputs, developer messages, image input and a new reasoning_effort parameter. o1 uses on average 60% fewer reasoning tokens than o1-preview.

Foundation Models OpenAIo1Reasoning

December 16, 2024 Medium

Pinokio 3.0: the local-AI browser-installer becomes customizable and adopts uv

In mid-December 2024 Pinokio, the open-source tool that installs and runs AI models locally without terminal commands, moves to the 3.0 series: customizable interface, uv as Python package manager, Playwright browser automation and per-app disk usage.

Local AI PinokioLocal AIInstaller

December 16, 2024 High

Google Veo 2 and an updated Imagen 3: video up to 4K, one week after Sora

On December 16, 2024 Google introduces Veo 2, a text-to-video model claiming output up to 4K and clips extendable to minutes, together with an improved Imagen 3 and the Whisk experiment. Veo 2 comes to VideoFX behind a waitlist.

Image & Video Gen GoogleDeepMindVeo 2

December 16, 2024 Medium

FlashInfer 0.2: customizable attention kernels for LLM serving

On December 16, 2024 the FlashInfer community releases 0.2 of its kernel library for LLM inference: FlashAttention-3 templates for Hopper GPUs, block- and vector-sparse attention, JIT-compiled attention variants, MLA decoding, and CUDAGraph and torch.compile compatibility.

AI Infrastructure FlashInferAttentionKV Cache

December 13, 2024 Medium

Pika 2.0: Scene Ingredients to compose videos from your own characters and objects

In mid-December 2024 Pika Labs releases Pika 2.0: its headline feature, Scene Ingredients, combines user-uploaded images (people, objects, clothing, settings) into one animated scene guided by a text prompt.

Image & Video Gen PikaVideo GenerationScene Ingredients

December 13, 2024 High

Google Agentspace: AI agents and enterprise search over data spread across many apps

On 13 December 2024 Google Cloud introduced Agentspace, in early access: NotebookLM Plus for enterprises, a multimodal search agent over company data with connectors to Confluence, Google Drive, Jira, SharePoint and ServiceNow, and expert agents to automate workflows.

Enterprise AI GoogleAgentspaceEnterprise Agents

December 11, 2024 Landmark

Gemini 2.0 Flash: Google opens the 'agentic era' and shows Astra, Mariner and Jules

On December 11, 2024 Google presents Gemini 2.0 Flash as an experimental release (multimodal, native tool use, image and audio output for selected partners) and three agent prototypes: Project Astra, Project Mariner and Jules.

Agents GoogleGemini 2.0Flash

December 9, 2024 High

Sora Turbo: ten months after the demo, OpenAI opens Sora to the public

On December 9, 2024, during the 12 Days of OpenAI, Sora Turbo arrives on sora.com for ChatGPT Plus and Pro subscribers: video from 480p to 1080p and up to 20 seconds, with Remix, Re-cut, Loop, Blend and Storyboard. The EU/EEA, UK and Switzerland are excluded at launch.

Image & Video Gen OpenAISoraSora Turbo

December 6, 2024 Medium

Llama 3.3 70B: Meta brings the 70B close to the 405B on several benchmarks

On December 6, 2024 Meta releases Llama 3.3 70B Instruct, text-only, 128k context. According to the model card it beats Llama 3.1 70B on almost every benchmark and tops Llama 3.1 405B on IFEval, MATH and GPQA Diamond.

Open Source Models MetaLlama 3.3Open Weights

December 5, 2024 High

InternVL 2.5: the first open-source multimodal model above 70% on MMMU

On December 5, 2024 OpenGVLab (Shanghai AI Lab) releases InternVL 2.5, a family of multimodal models from 1B to 78B; according to the paper the 78B version is the first open-source MLLM to surpass 70% on MMMU, competitive with GPT-4o and Claude 3.5 Sonnet.

Multimodal AI VLMMMMUChain-of-Thought

December 5, 2024 High

12 Days of OpenAI: o1 and ChatGPT Pro, Sora, Projects, o3 preview

From December 5 to 20, 2024 OpenAI announced one release per weekday: full o1 and the $200/month ChatGPT Pro plan on day one, Sora for Plus and Pro users, Canvas for everyone, Projects, ChatGPT search for free users, o1 in the API, and on the last day the o3 and o3-mini preview.

Foundation Models OpenAIChatGPTo1

December 3, 2024 High

HunyuanVideo: Tencent releases the weights of a 13-billion-parameter video model

On December 3, 2024 Tencent releases weights and inference code for HunyuanVideo, a text-to-video model with over 13B parameters that generates 129-frame 720p clips. In Tencent's evaluation with professional raters it ranks ahead of Runway Gen-3 alpha and Luma 1.6. The license explicitly excludes the European Union.

Image & Video Gen HunyuanVideoTencentopen weights

December 3, 2024 High

Amazon Launches the Nova Model Family at re:Invent

Amazon enters the foundation model race with Nova Micro, Lite and Pro on Bedrock, plus Nova Canvas for images and Nova Reel for video, and a larger Nova Premier announced for Q1 2025. Amazon states Nova Pro is equal to or better than GPT-4o on 17 of 20 benchmarks.

Enterprise AI

November 2024

November 26, 2024 Medium

SmolVLM: Hugging Face's 2B VLM built for a small memory footprint

Hugging Face releases SmolVLM, a 2B-parameter vision-language model built on SmolLM2 1.7B that compresses images into few tokens and needs about 5 GB of GPU RAM. Checkpoints, datasets and recipes under Apache 2.0.

Multimodal AI Edge AIVLMSmall Model

November 26, 2024 Medium

OLMo 2: Ai2 releases fully open 7B and 13B models competitive with Llama 3.1

Ai2 releases OLMo 2 (7B and 13B, trained on up to 5T tokens): the 7B outperforms Llama 3.1 8B, the 13B outperforms Qwen 2.5 7B with fewer training FLOPs. Weights, pretraining datasets and the Tülu 3 post-training recipe are published.

Open Source Models OLMo 2Ai2open source

November 25, 2024 High

Model Context Protocol: the open standard to connect LLMs and data

On November 25, 2024 Anthropic open-sources the Model Context Protocol (MCP), a JSON-RPC-based standard for connecting AI assistants to external data and tools without a custom integration for every source.

AI Infrastructure AnthropicMCPModel Context Protocol

November 25, 2024 Medium

llama.cpp: speculative decoding with a draft model comes to llama-server

On November 25, 2024 Georgi Gerganov merges PR #10455: llama-server, the llama.cpp HTTP server, supports speculative decoding with a draft model. In the PR's tests, Qwen2.5-Coder 32B with a 0.5B draft goes from 29 to 47 tokens/s on an RTX 3090 (about 1.6x).

Local AI llama.cppllama-serverSpeculative Decoding

November 21, 2024 Medium

Ai2's Tülu 3: a fully open post-training recipe

The Allen Institute for AI (Ai2) releases Tülu 3, 8B and 70B models built on Llama 3.1 base with SFT, DPO and the new RLVR; it publishes data, code, recipes and evaluations. According to Ai2 it surpasses the Instruct versions of Llama 3.1, Qwen 2.5 and Mistral.

Open Source Models AI2Allen InstituteTulu 3

November 19, 2024 High

UK and US AISI publish the pre-deployment evaluation of the new Claude 3.5 Sonnet

On November 19, 2024 the UK and US AI Safety Institutes publish their first joint pre-deployment evaluation: the upgraded Claude 3.5 Sonnet, tested on biology, cyber, software and AI development and safeguard efficacy, compared with the previous Sonnet, o1-preview and GPT-4o.

AI Security AISIUK AI Safety InstituteUS AI Safety Institute

November 19, 2024 Medium

Suno v4: cleaner audio, Remaster, Covers, Personas and lyrics with ReMi

On November 19, 2024 Suno launches its v4 model in beta for Pro and Premier subscribers: cleaner audio, sharper lyrics and more dynamic song structures, together with Remaster, the ReMi lyrics model, Covers and Personas.

Voice & Audio SunoMusic GenerationAudio

November 18, 2024 Medium

Pixtral Large: Mistral brings vision to the 124-billion size

On November 18, 2024 Mistral unveils Pixtral Large, a 124B open-weights multimodal model (123B decoder + 1B vision encoder), under a research or commercial license. The same announcement brings Mistral Large 24.11.

Multimodal AI MistralPixtral LargeVision

November 13, 2024 Medium

Windsurf: Codeium launches its editor with the Cascade agentic flow

Codeium makes Windsurf Editor available, a VS Code fork built around "AI flows": Cascade reasons over the codebase, edits and creates files, runs commands and stays aware of what the developer does. The editor and many Cascade features are free at launch.

AI Coding CodeiumWindsurfAI Coding

November 12, 2024 High

Qwen2.5-Coder-32B: the open model that closes in on GPT-4o for code

On November 12, 2024 Alibaba completes the Qwen2.5-Coder family with the 32B-Instruct (Apache 2.0): 92.7 on HumanEval vs GPT-4o's 92.1 and 73.7 on Aider, but it stays behind GPT-4o on LiveCodeBench (31.4 vs 34.6). 32K native context, 128K with YaRN.

AI Coding Qwen2.5-Coderopen sourcecode generation

November 6, 2024 Medium

NVIDIA at CoRL 2024: Isaac Lab generally available, six Project GR00T workflows and the Cosmos tokenizer

On November 6, 2024, at the Conference on Robot Learning in Munich, NVIDIA makes Isaac Lab generally available, announces six Project GR00T workflows for humanoid robots (coming soon) and open-sources the Cosmos tokenizer for images and video.

Robotics NVIDIAProject GR00TIsaac Lab

November 6, 2024 High

Browser Use: the open-source library that lets LLMs drive a browser

On November 6, 2024 Browser Use ships its first release (0.1.0): an MIT-licensed Python library that lets an LLM carry out tasks on real websites by detecting the page's interactive elements, handling multiple tabs and supporting vision models. Built on Selenium at launch, it moves to Playwright at the end of November (0.1.10).

Agents Browser Usebrowser automationPlaywright

November 6, 2024 Medium

FLUX1.1 [pro] Ultra and Raw: up to 4MP images in about 10 seconds and a natural photographic look

Black Forest Labs adds to FLUX1.1 [pro] an Ultra mode (up to 4 megapixels, 4x the standard resolution, ~10 seconds per image, $0.06 per image) and a Raw mode with a less synthetic look resembling candid photography. API only.

Image & Video Gen Black Forest LabsFLUXImage Generation

November 6, 2024 Medium

Ollama 0.4 brings Llama 3.2 Vision locally with one command

With version 0.4 Ollama supports Meta's Llama 3.2 Vision: 'ollama run llama3.2-vision' runs the 11B model locally (at least 8 GB of VRAM), the 90B variant needs at least 64. Images go in the prompt or in the API images field.

Local AI Ollamavisionmultimodal

November 4, 2024 High

Magentic-One: Microsoft Research's generalist multi-agent system

Microsoft Research presents Magentic-One, built on AutoGen: an Orchestrator that plans, tracks progress and re-plans directs four agents (WebSurfer, FileSurfer, Coder, ComputerTerminal). On GAIA, AssistantBench and WebArena it is statistically competitive with the state of the art (38%, 27.7%, 32.8%). Released as open source together with AutoGenBench.

Agents Magentic-Onemulti-agentMicrosoft Research

November 1, 2024 Medium

Roo Cline: the Cline fork that will become Roo Code is born

In early November 2024 a small team publishes Roo Cline, a fork of the Cline coding agent for VS Code: its first additions are automatic approval of writes and commands, a command allow-list and support for .clinerules and .cursorrules.

AI Coding Roo ClineRoo CodeCline

October 2024

October 31, 2024 High

Physical Intelligence's π0: a generalist robot model for different robots

On October 31, 2024 startup Physical Intelligence presents π0, a vision-language-action model built on PaliGemma (3B) with a flow-matching action expert, pre-trained on about 10,000 hours of data from 7 robot configurations and 68 tasks, able to fold laundry and bus tables.

Robotics Physical IntelligencePi ZeroVLA

October 29, 2024 Medium

GitHub Spark: personal micro-apps described in natural language

At GitHub Universe 2024, GitHub Next unveils Spark as a waitlisted technical preview: describe a small app in natural language and Spark runs it in a live preview, with managed key-value storage and no deployment step.

AI Coding GitHub SparkGitHub Nextnatural language

October 29, 2024 Medium

GitHub Universe 2024: Copilot Workspace comes to pull requests

At GitHub Universe 2024 Copilot Workspace arrives inside pull requests in public preview (waitlisted): it is used to refine, test and land Copilot-generated code review suggestions. At the same event Copilot goes multi-model.

AI Coding GitHubCopilotWorkspace

October 23, 2024 Medium

ElevenLabs Voice Design: a new voice from a text description

In late October 2024 ElevenLabs launches Voice Design: you describe in words the voice you have in mind (age, accent, tone, character) and the system generates a new one, with no source audio.

Voice & Audio ElevenLabsVoice DesignText-to-Voice

October 22, 2024 Medium

Stable Diffusion 3.5: 8.1B-parameter Large, 4-step Large Turbo, then Medium

Stability AI releases Stable Diffusion 3.5 Large (8.1B parameters) and Large Turbo, a distilled version that generates in 4 steps; Medium (2.5B) follows on October 29. Weights on Hugging Face under the Stability AI Community License: free commercial use below $1 million in annual revenue.

Image & Video Gen Stable Diffusion 3.5Stability AIopen weights

October 22, 2024 High

Computer Use: Claude learns mouse and keyboard

On October 22, 2024 Anthropic releases 'Computer Use' in public beta on the upgraded Claude 3.5 Sonnet: the model looks at screenshots, moves the cursor, clicks and types. On OSWorld it scores 14.9% (screenshot-only) versus 7.8% for the next-best system.

Agents AnthropicClaudeComputer Use

October 21, 2024 High

Microsoft: autonomous agents in Copilot Studio and ten agents for Dynamics 365

On October 21, 2024 Microsoft announces the ability to build autonomous agents in Copilot Studio, which start from events or on a schedule without human input, in public preview from Ignite 2024. It also announces ten prebuilt autonomous agents for Dynamics 365, including the Sales Qualification Agent.

Enterprise AI MicrosoftCopilot StudioAutonomous Agents

October 15, 2024 Medium

Anthropic updates its Responsible Scaling Policy (v2.0): Capability Thresholds and Required Safeguards

On October 15, 2024 Anthropic publishes version 2.0 of its Responsible Scaling Policy: capability thresholds for CBRN weapons and autonomous AI research, tied to ASL safety standards, new capability and safeguard assessments, and Jared Kaplan as Responsible Scaling Officer.

AI Security AnthropicRSPSafety

October 14, 2024 High

Adobe Firefly Video Model: 'commercially safe' AI video in beta, with Generative Extend in Premiere Pro

At Adobe MAX (October 14, 2024) Adobe presents the Firefly Video Model in limited public beta: Text to Video and Image to Video in the Firefly web app and Generative Extend in the Premiere Pro beta. Adobe calls it the first publicly available video model designed to be commercially safe.

Image & Video Gen Adobe Fireflyvideo generationcommercially safe

October 11, 2024 Medium

OpenAI Swarm: experimental, educational library for agents with handoffs

OpenAI publishes Swarm on GitHub, a lightweight Python library, explicitly experimental and educational, for coordinating multiple agents through routines and handoffs on top of the Chat Completions API. It is later replaced by the OpenAI Agents SDK.

Agents OpenAISwarmAgents

October 9, 2024 Landmark

2024 Chemistry Nobel to Baker, Hassabis and Jumper: protein design and prediction

The Royal Swedish Academy of Sciences awards half of the 2024 Chemistry Nobel to David Baker for computational protein design and the other half to Demis Hassabis and John Jumper of Google DeepMind for protein structure prediction with AlphaFold2.

Foundation Models Nobel PrizeAlphaFoldHassabis

October 8, 2024 High

GR-2: ByteDance Research pre-trains a robot on 38 million internet video clips

ByteDance Research presents GR-2, a generative video-language-action model pre-trained on about 38 million video clips (over 50 billion tokens) and then fine-tuned on robot trajectories. It reaches a 97.7% average success rate on 105 manipulation tasks and generalizes to unseen backgrounds, environments and objects.

Robotics GR-2ByteDancevideo pretraining

October 8, 2024 Landmark

2024 Nobel Prize in Physics to Hopfield and Hinton for artificial neural networks

The Royal Swedish Academy of Sciences awards the 2024 Nobel Prize in Physics to John Hopfield and Geoffrey Hinton for foundational discoveries and inventions that enable machine learning with artificial neural networks.

Foundation Models Nobel PrizeHintonHopfield

October 8, 2024 Medium

F5-TTS: zero-shot speech synthesis with flow matching, open code and models

A Shanghai Jiao Tong University team releases F5-TTS: a non-autoregressive TTS based on flow matching and a Diffusion Transformer, with no duration model or phoneme alignment. Trained on 100K hours, RTF 0.15; MIT code, CC-BY-NC weights.

Voice & Audio F5-TTSFlow MatchingVoice Cloning

October 6, 2024 Medium

Gradient Routing: masking gradients to localize capabilities in a neural network

An arXiv paper from October 6, 2024 (Cloud, Goldman-Wetzler, Wybitul, Miller, Turner) proposes gradient routing: masks on gradients decide which parameters each data point updates, confining capabilities to network regions that can later be removed.

AI Security Gradient RoutingInterpretabilityUnlearning

October 4, 2024 High

Meta Presents Movie Gen: 16-Second Video with Synchronized Audio

On 4 October 2024 Meta presents Movie Gen, a research family of media models: a 30B-parameter video model that generates up to 16 seconds at 16 fps, a 13B audio model for synchronized sound, plus personalized generation and text-based editing. No public release.

Image & Video Gen

October 3, 2024 High

Bolt.new: full-stack app from a prompt, in the browser, no install needed

On October 3, 2024 (the date reported by launch trackers) StackBlitz launches Bolt.new: from a prompt it generates, runs and edits full-stack web apps directly in the browser, combining StackBlitz's WebContainers with Claude 3.5 Sonnet.

AI Coding Full-Stack GenerationBrowser IDEWebContainers

October 3, 2024 Medium

OpenAI Canvas: a side-by-side editor for writing and code in ChatGPT

On October 3, 2024 OpenAI introduces Canvas in beta: a separate window where ChatGPT and the user work on the same text or code, with targeted edits and shortcuts. Built on GPT-4o, it starts with Plus and Team users.

AI Coding

October 2, 2024 Medium

n8n 1.62.1: app nodes become tools for the AI Agent

With release 1.62.1 (October 2, 2024) n8n makes more nodes usable as tools for the Tools AI Agent and adds the $fromAI() function, which lets the model pass dynamic parameters to connected tools.

Agents n8nNo-CodeAutomation

September 2024

September 30, 2024 Medium

Whisper large-v3-turbo locally: 809M parameters, 8x faster, with whisper.cpp and faster-whisper

OpenAI released large-v3-turbo: 809M parameters against large-v3's 1.55B, obtained by cutting the decoder from 32 layers to 4 and fine-tuning for two epochs on multilingual transcription data only. Roughly 8x faster and about 6 GB of VRAM, with quality close to large-v2 but no translation support and clear degradation on Thai and Cantonese.

Voice & Audio

September 27, 2024 High

Emu3: a single transformer for text, images and video using only next-token prediction

On September 27, 2024 BAAI's Emu3 team publishes Emu3: images, text and video are converted into discrete tokens and a single 8B transformer, trained from scratch with next-token prediction only, handles both generation and understanding.

Multimodal AI Unified ModelAutoregressiveNext-Token Prediction

September 26, 2024 Medium

torchao: native quantization and sparsity in PyTorch

The PyTorch team officially launches torchao, a library that makes models faster and smaller with low-bit dtypes, quantization and sparsity, written mostly in readable PyTorch. Headline figure: 97% faster inference on Llama 3 8B with INT4 weight-only quantization.

AI Infrastructure torchaoquantizationINT4

September 25, 2024 High

Molmo: Ai2's open VLMs that point at objects

On September 25, 2024 Ai2 releases Molmo, a family of open vision-language models (1B, 7B, 72B) that can point at precise locations in images. The 72B beats Claude 3.5 Sonnet and Gemini 1.5 Pro on academic benchmarks and ranks second only to GPT-4o in human evaluation.

Multimodal AI VLMOpen SourcePointing

September 25, 2024 High

Llama 3.2: Meta brings vision and edge to open models

On September 25, 2024, at Meta Connect, Meta releases Llama 3.2 in 4 sizes: 1B and 3B text-only for edge and mobile devices, 11B and 90B multimodal with image input. They are the first Llama models with vision.

Open Source Models MetaLlama 3.2Multimodal

September 25, 2024 Medium

Llama Stack: Meta proposes standard interfaces and distributions for building apps on Llama models

Alongside Llama 3.2, on September 25, 2024 Meta introduces the first Llama Stack distributions: a standardized interface for toolchain components, with a reference implementation for inference, tool use and RAG and distributions for cloud, on-prem, single-node and on-device.

AI Infrastructure MetaLlama StackOpen Source

September 24, 2024 High

Figma First Draft: prompt-to-UI generation returns in beta after Make Designs was pulled

On September 24, 2024 Figma reintroduces prompt-based interface generation in limited beta, now called First Draft. The feature, launched as Make Designs at June's Config, had been disabled in July because it produced mockups too similar to existing apps.

Enterprise AI FigmaDesign AIUI Generation

September 19, 2024 High

Qwen 2.5: Alibaba's open family spans 0.5B to 72B, with Coder and Math variants

Alibaba releases Qwen 2.5 in seven sizes (0.5B–72B) trained on up to 18 trillion tokens, together with Qwen2.5-Coder (1.5B and 7B) and Qwen2.5-Math (1.5B, 7B, 72B). Apache 2.0 license for every size except 3B and 72B.

Open Source Models AlibabaQwenOpen Source

September 19, 2024 Medium

OCI Generative AI adds Llama 3.1 70B and 405B, with dedicated AI clusters and LoRA fine-tuning

According to the September 19, 2024 release notes, Oracle's OCI Generative AI service supports Meta's Llama 3.1 70B and 405B: 405B on-demand in Chicago and on dedicated AI clusters in three more regions, 70B also with LoRA fine-tuning.

Enterprise AI OracleOCIGenerative AI

September 19, 2024 Medium

Kling 1.5: 1080p video in professional mode and Motion Brush

Kuaishou updates the Kling video generator to version 1.5: video up to 1080p in professional mode, a new Motion Brush to steer the motion of individual elements, and an overall 95% improvement according to the company's internal tests.

Image & Video Gen KuaishouKlingVideo Generation

September 18, 2024 Medium

HubSpot Breeze: copilot, four agents and data enrichment in the CRM (INBOUND 2024)

At INBOUND 2024 HubSpot unveils Breeze, the AI embedded in its platform: Breeze Copilot, four Breeze Agents (Content, Social Media, Prospecting, Customer) and Breeze Intelligence for data enrichment from over 200 million profiles.

Enterprise AI HubSpotCRMAI Agents

September 18, 2024 High

Copilot Autofix free on all public repositories: CodeQL finds the vulnerability, Copilot proposes the fix

From September 18, 2024 Copilot Autofix for CodeQL alerts is free on all public repositories, after general availability for GitHub Advanced Security customers on August 14. In beta data, fixes were more than three times faster.

AI Coding SecurityGitHubCodeQL

September 17, 2024 Medium

1X World Model: a simulator learned from EVE robot data to evaluate policies

1X Technologies presents a world model trained on thousands of hours of video and action data from its EVE humanoid: given the current state and a sequence of actions, it generates video of what will happen next. It serves as a learned simulator for evaluating robot policies. A public challenge launches alongside, with over 100 hours of data released.

Robotics 1Xworld modelhumanoid

September 17, 2024 High

Pixtral 12B: Mistral's multimodal model with a vision encoder trained from scratch

Mistral releases Pixtral 12B under Apache 2.0: a 400M-parameter vision encoder trained from scratch, a 12B decoder based on Mistral Nemo, images at native resolution and a 128K-token context with multiple images.

Multimodal AI PixtralMistralVision Encoder

September 12, 2024 Landmark

o1: the model that 'thinks before answering'

OpenAI releases o1-preview and o1-mini, models trained with reinforcement learning to reason through a long internal chain of thought. On competition math, coding and science they clearly beat GPT-4o.

Foundation Models OpenAIo1Reasoning

September 12, 2024 Medium

Fish Speech 1.4: open-weights TTS trained on 700k hours in 8 languages

Fish Audio releases Fish Speech 1.4, a text-to-speech model with voice cloning trained on about 700k hours of audio in 8 languages. Code under BSD-3-Clause, weights under CC-BY-NC-SA 4.0 (non-commercial use only).

Voice & Audio Fish SpeechTTSVoice Cloning

September 12, 2024 High

ALOHA Unleashed: a two-armed robot that ties shoelaces and hangs shirts

On 12 September 2024 Google DeepMind presents ALOHA Unleashed: a policy trained with Diffusion Policy on teleoperated ALOHA 2 demonstrations that ties shoelaces, hangs a shirt, replaces another robot's finger, inserts gears and tidies kitchen items.

Robotics DeepMindALOHA UnleashedDiffusion Policy

September 11, 2024 High

Hume AI EVI 2: a voice-to-voice model that adapts tone and style to the listener

On September 11, 2024 Hume AI releases EVI 2 in beta, a voice-to-voice foundation model that merges speech and language in one system: subsecond responses, adjustable voices without cloning, available in the app and via API.

Voice & Audio Hume AIEVI 2Empathic Voice Interface

September 4, 2024 Medium

Anthropic launches the Claude Enterprise plan: 500K context, GitHub integration, SSO, and role-based permissions

Anthropic introduces Claude Enterprise: a 500K-token context window, more usage capacity, a native GitHub integration in beta, SSO with domain capture, and role-based permissions; SCIM and audit logs announced for the following weeks.

Enterprise AI AnthropicClaudeEnterprise

August 2024

August 29, 2024 High

Qwen2-VL: dynamic resolution, 20+ minute video and open weights for 2B and 7B

On August 29, 2024 Alibaba's Qwen team unveils Qwen2-VL: images at arbitrary resolution, understanding of videos over 20 minutes, agent capabilities. 2B and 7B ship with Apache 2.0 open weights, 72B via API only.

Multimodal AI Qwen2-VLDynamic ResolutionM-RoPE

August 27, 2024 Medium

Cerebras Inference: LLM inference on the wafer-scale WSE-3 chip

On August 27, 2024 Cerebras launches an inference service on WSE-3 chips, claiming 1,800 tokens/s on Llama 3.1 8B and 450 tokens/s on Llama 3.1 70B, 20x faster than GPU-based solutions in hyperscale clouds, at 10 and 60 cents per million tokens.

AI Infrastructure CerebrasWSE-3Inference

August 22, 2024 Medium

Cursor 0.40: Composer on by default, new chat and new Tab model

On August 22, 2024 Cursor 0.40 turns on Composer (multi-file editing, Cmd+I) by default for Pro and Business users, adds Composer Projects in beta, redesigns the chat and introduces a new Cursor Tab model with TypeScript auto-imports.

AI Coding IDEMulti-file EditingComposer

August 22, 2024 Medium

LM Studio 0.3.0: chat with documents, structured output and multiple models served in parallel

On August 22, 2024 LM Studio 0.3.0 adds chat with documents (with RAG for long files), JSON-schema structured output for any local model, parallel serving of multiple models, a network-reachable server, hardware-based auto-configuration, new themes and chat folders.

Local AI LM StudioRAGStructured Output

August 21, 2024 Medium

Ideogram 2.0: styles, color palettes, iOS app and API

Ideogram releases its 2.0 model with Realistic, Design, 3D and Anime styles, color palette control, an iOS app and a beta API; the Design style targets more accurate text rendering.

Image & Video Gen Ideogram 2.0text renderingtypography

August 21, 2024 High

CrossFormer: one policy for manipulation, navigation, locomotion and aviation

Researchers from UC Berkeley and CMU present CrossFormer, a transformer policy trained on 900K trajectories from very different robots: with the same weights it controls single- and dual-arm manipulators, wheeled robots, quadrupeds and drones, without hand-aligning action spaces.

Robotics CrossFormercross-embodimentfoundation model

August 20, 2024 Medium

Zed AI: the Zed editor launches its hosted AI service with Anthropic

Zed introduces Zed AI, a hosted service powered by Claude 3.5 Sonnet and built in collaboration with Anthropic: a text-based assistant panel, slash commands to insert context, and inline code transformations. Free during the launch period.

AI Coding ZedEditorRust

August 15, 2024 Medium

ADAS: a meta-agent that programs new AI agents in code

Shengran Hu, Cong Lu and Jeff Clune (University of British Columbia, Vector Institute) propose ADAS and the Meta Agent Search algorithm: a GPT-4-based meta-agent writes new agents in Python, evaluates them and archives them. The discovered agents beat hand-designed ones, for example +13.6 F1 points on DROP and +14.4% on MGSM.

Agents ADASMeta Agent Searchautomated design

August 14, 2024 Low

Dify 0.7.0: conversation variables and the Variable Assigner node in workflows

Dify 0.7.0 adds conversation variables and the Variable Assigner node to workflows, giving LLM apps memory across turns, plus new models, new tools and Elasticsearch as a vector database.

Agents DifyWorkflowChatflow

August 14, 2024 Medium

LLM Compressor: one library to quantize and sparsify models for serving with vLLM

Neural Magic releases LLM Compressor 0.1, an open source library that brings GPTQ, SmoothQuant, SparseGPT and FP8 quantization into a single tool for creating compressed models to serve with vLLM.

AI Infrastructure LLM CompressorNeural MagicQuantizzazione

August 13, 2024 Medium

SWE-bench Verified: OpenAI and the SWE-bench authors clean up the coding agent benchmark

OpenAI, together with the SWE-bench authors, releases SWE-bench Verified: 500 tasks from the original test set checked by human developers, superseding SWE-bench and SWE-bench Lite as evaluation sets.

AI Coding OpenAISWE-benchEvaluation

August 8, 2024 Medium

Parler-TTS v1: Hugging Face releases Mini and Large, voices controlled by a text description

Hugging Face publishes Parler-TTS Mini v1 (880M) and Large v1 (about 2.2B), trained on 45,000 hours of audiobooks: the voice is described in words (gender, speed, pitch, noise). Open weights, data and training code, Apache 2.0 license.

Voice & Audio Parler TTSHuggingFaceControllable TTS

August 6, 2024 High

CogVideoX: Zhipu AI and Tsinghua release an open-weight text-to-video model

On August 6, 2024 Zhipu AI and Tsinghua's THUDM group publish CogVideoX-2B with its 3D causal VAE; CogVideoX-5B follows on August 27. Both generate 6-second clips at 720x480 and 8 fps. The 2B moves to Apache 2.0, while the 5B stays under the CogVideoX License.

Image & Video Gen CogVideoXopen sourcetext-to-video

August 6, 2024 High

Figure 02: Figure AI's second humanoid, with voice dialogue built on models developed with OpenAI

On August 6, 2024 Figure AI unveils Figure 02: three times the onboard compute, six cameras, 16-degree-of-freedom hands, a 2.25 kWh battery and speech-to-speech conversation using models developed with OpenAI. The robot was tested at BMW's Spartanburg plant.

Robotics Figure AIFigure 02OpenAI

August 6, 2024 Medium

OpenAI launches Structured Outputs in the API

On August 6, 2024 OpenAI adds Structured Outputs to its API: with strict mode, responses are constrained to match a developer-supplied JSON Schema. gpt-4o-2024-08-06 scores 100% on OpenAI's schema-following eval, versus under 40% for gpt-4-0613.

Foundation Models

August 1, 2024 High

FLUX.1 (Black Forest Labs): 12B parameters, flow matching, three variants

Black Forest Labs, the new startup from the researchers behind Latent Diffusion and Stable Diffusion, launches FLUX.1: 12-billion-parameter text-to-image models trained with flow matching, in three variants (pro via API, dev open-weight non-commercial, schnell Apache 2.0).

Image & Video Gen FluxBlack Forest LabsFlow Matching

July 2024

July 30, 2024 High

ChatGPT Advanced Voice Mode: alpha starts for a small group of Plus users

On July 30, 2024 OpenAI starts the alpha of ChatGPT's Advanced Voice Mode, built on GPT-4o's native audio, for a small group of Plus subscribers. Four preset voices, no Sky, rollout to all Plus users planned for the fall.

Voice & Audio OpenAIAdvanced Voice ModeChatGPT

July 26, 2024 Medium

NIST AI 600-1: the AI RMF profile for generative AI

NIST publishes AI 600-1, a cross-sectoral profile of and companion resource for the AI Risk Management Framework covering generative AI, issued under Executive Order 14110: it identifies 12 risks unique to or exacerbated by GenAI and proposes actions to manage them.

AI Security NIST AI 600-1generative AIrisk profile

July 26, 2024 Medium

AnythingLLM v1.0.0: Mintplex Labs starts versioned releases

On July 26, 2024 Mintplex Labs publishes the v1.0.0 tag of AnythingLLM on GitHub, the open-source (MIT) application for querying your own documents with an LLM. It is not a new feature: the release starts a formal release cycle and collects in its changelog the work accumulated since 2023.

Local AI AnythingLLMRAGself-hosted

July 25, 2024 High

AlphaProof and AlphaGeometry 2: silver-medal level at the International Mathematical Olympiad

DeepMind announces that AlphaProof (on Lean) and AlphaGeometry 2 solved 4 of the 6 problems at the 2024 International Mathematical Olympiad, scoring 28 of 42 points: silver-medal level, one point below the gold threshold.

Foundation Models DeepMindAlphaProofAlphaGeometry 2

July 23, 2024 Landmark

Llama 3.1 405B: open-weight models reach the frontier

Meta releases Llama 3.1 in 8B, 70B and 405B versions with a 128K-token context and the Llama 3.1 Community license: according to Meta the 405B is competitive with GPT-4o and Claude 3.5 Sonnet.

Open Source Models MetaLlama 3.1405B

July 22, 2024 Medium

Flowise 2.0: Sequential Agents with loops, conditional branches and human-in-the-loop

Flowise 2.0 introduces Sequential Agentic Workflows: chained agents, loops back to earlier steps, human approval before sensitive actions and conditional branches, with seven example flows.

Agents FlowiseVisual BuilderLow-Code

July 18, 2024 Medium

Mistral NeMo 12B: 128k context, Apache 2.0, drop-in replacement for Mistral 7B

Mistral AI and NVIDIA release Mistral NeMo, a 12B model with up to 128k tokens of context, Apache 2.0 license, the new Tekken tokenizer and FP8 inference; designed as a drop-in replacement for Mistral 7B.

Foundation Models Mistral NeMoNVIDIAOpen Weights

July 18, 2024 High

GPT-4o mini: OpenAI's cheap model takes over from GPT-3.5 Turbo

OpenAI ships GPT-4o mini at $0.15/$0.60 per million tokens, more than 60% cheaper than GPT-3.5 Turbo, with 82% MMLU and 128K context. In ChatGPT it replaces GPT-3.5 for Free, Plus and Team users.

Foundation Models OpenAIGPT-4o miniCost Efficiency

July 17, 2024 High

Open WebUI: Tools and Functions turn the frontend into a plugin platform

Between June and July 2024 Open WebUI adds Tools (native Python function calling, v0.3.3), Filter and Pipe Functions (v0.3.6) and, on July 17 with v0.3.9, Actions: Python plugins running directly inside the Open WebUI server.

Local AI Open WebUIFunction CallingTools

July 11, 2024 High

FlashAttention-3: attention 1.5-2x faster than FA2 on H100, up to 740 TFLOPS in FP16

Tri Dao and researchers from Colfax, Meta and NVIDIA publish FlashAttention-3: it exploits the Tensor Core and TMA asynchrony of Hopper GPUs and the FP8 format. On H100 it is 1.5-2x faster than FlashAttention-2, up to 740 TFLOPS in FP16 (75% utilization) and about 1.2 PFLOPS in FP8.

AI Infrastructure FlashAttention-3H100Hopper

July 7, 2024 Medium

CosyVoice: Alibaba's open-source multilingual zero-shot speech synthesis

Alibaba's Speech Lab publishes CosyVoice, zero-shot speech synthesis in Chinese, English, Japanese, Korean and Cantonese: an LLM generates semantic speech tokens, a flow matching model turns them into audio. 300M models and code are open source under Apache 2.0, as part of the FunAudioLLM family.

Voice & Audio CosyVoiceAlibabaFunAudioLLM

July 3, 2024 High

Moshi: Kyutai unveils a real-time voice AI, testable online right away

On July 3, 2024 in Paris the non-profit lab Kyutai unveils Moshi, a full-duplex voice AI prototype (announced latency around 160 ms) built in 6 months by 8 researchers on a 7B LLM called Helium. Online demo the same day, code and weights promised soon.

Voice & Audio KyutaiMoshiVoice

July 2, 2024 Medium

Supermaven 1.0: the Babble model brings autocomplete to a 1-million-token context

Supermaven, the code completion tool founded by Jacob Jackson (creator of TabNine), reaches version 1.0: the new Babble model is 2.5x larger than its predecessor and expands the context window from 300,000 to 1 million tokens. In-editor chat ships too.

AI Coding SuperMavenBabbleautocomplete

July 1, 2024 Medium

Agentless: no agent loop, best open-source result on SWE-bench Lite

A University of Illinois Urbana-Champaign group publishes Agentless: it localizes the problem, generates patches and filters them with tests, without an agent choosing its moves. With GPT-4o it resolves 27.33% of SWE-bench Lite at $0.34 per issue, the best result among open-source tools at the time.

Agents AgentlessSWE-benchcode repair

June 2024

June 27, 2024 High

Gemma 2: the second generation of Google's open-weights models

Google releases Gemma 2 in 9B and 27B sizes: alternating local/global attention, logit soft-capping, a 9B trained by distillation. Google says the 27B is competitive with models more than twice its size; a 2.6B is announced as coming.

Open Source Models GoogleGemma 2Open Weights

June 24, 2024 High

Mooncake: the KV-cache-centric disaggregated architecture serving Kimi

Moonshot AI publishes the technical report for Mooncake, the serving platform behind the Kimi chatbot: prefill and decode run on separate clusters and the KV cache is spread across the CPU, DRAM and SSD of GPU servers. Up to 525% more throughput than the baseline in simulated scenarios, 75% more requests handled under real workloads.

AI Infrastructure Mooncakedisaggregated inferenceprefill-decode

June 20, 2024 High

Claude 3.5 Sonnet: the mid-tier that beats Claude 3 Opus

Anthropic releases Claude 3.5 Sonnet: it beats Claude 3 Opus (the previous flagship) on stated benchmarks, at Sonnet pricing ($3/$15 per million tokens) and twice Opus's speed. Claude.ai gets a preview of 'Artifacts', a side panel for code and documents.

Foundation Models AnthropicClaude 3.5 SonnetArtifacts

June 17, 2024 High

DeepSeek-Coder-V2: an open-weight MoE code model close to GPT-4 Turbo

DeepSeek releases Coder-V2 in 16B (2.4B active) and 236B (21B active) MoE variants, built by continuing DeepSeek-V2 pretraining on 6T additional tokens: 338 languages and 128K context. It beats GPT-4 Turbo on HumanEval, MBPP+, Aider and MATH, but trails on LiveCodeBench and SWE-Bench.

AI Coding DeepSeek-Coder-V2MoEopen weights

June 17, 2024 High

Runway Gen-3 Alpha: a new generation of video models with clips up to 10 seconds

On June 17, 2024 Runway unveils Gen-3 Alpha, the first model of a new generation trained jointly on video and images: better fidelity, consistency and motion than Gen-2, 5- or 10-second clips, paid subscribers first.

Image & Video Gen RunwayGen-3Video Generation

June 15, 2024 High

Florence-2: a single visual model for captioning, detection, segmentation, and OCR

Microsoft publishes on Hugging Face, under the MIT license, the weights of Florence-2 (0.23B and 0.77B): a vision foundation model that handles captioning, object detection, grounding, segmentation and OCR with a single prompt-driven sequence-to-sequence architecture.

Multimodal AI MicrosoftFlorence-2Vision Foundation Model

June 15, 2024 High

HumanPlus: a humanoid that shadows a person in real time and learns autonomous skills

Stanford researchers present HumanPlus: a policy trained with RL in simulation on about 40 hours of human motion data lets a 33-DoF humanoid shadow an operator's body and hands in real time, captured with a single RGB camera. The demonstrations collected this way train autonomous skills with 60-100% success using up to 40 demonstrations.

Robotics HumanPlushumanoidshadowing

June 14, 2024 Medium

Nemotron-4 340B: NVIDIA's model family for generating synthetic training data

NVIDIA releases Nemotron-4 340B in three versions, Base, Instruct and Reward, with downloadable weights under the NVIDIA Open Model License. They are designed as a pipeline for generating synthetic data to train other models: over 98% of the data used to align the Instruct model itself is synthetic.

Foundation Models Nemotron-4NVIDIAsynthetic data

June 13, 2024 Medium

OpenVLA: an open-source 7B Vision-Language-Action model for robotics

On June 13, 2024 researchers from Stanford, UC Berkeley, Toyota Research Institute, Google DeepMind, Physical Intelligence and MIT publish OpenVLA: a 7B model trained on 970,000 robot demonstrations that beats the closed RT-2-X (55B) by 16.5% in absolute success rate across 29 tasks.

Robotics OpenVLAStanfordBerkeley

June 13, 2024 Medium

Leonardo AI Phoenix: the first in-house foundation model, built around prompt adherence

In mid-June 2024 Leonardo AI releases Phoenix in preview, its first in-house foundation model: it focuses on prompt adherence, legible text in images and Prompt Enhance.

Image & Video Gen Leonardo AIPhoenixPrompt Adherence

June 12, 2024 Medium

Luma Dream Machine: video generation open to everyone, with a free tier

Luma AI makes Dream Machine public, a video generator from text or images: 5-second clips, web access with a Google account and 30 free videos per month, while OpenAI's Sora is still not accessible.

Image & Video Gen LumaDream MachineVideo Generation

June 12, 2024 Medium

Databricks Mosaic AI at Data + AI Summit: Agent Framework, Agent Evaluation and tooling for compound AI systems

At Data + AI Summit 2024 Databricks unveils new Mosaic AI capabilities for building compound AI systems on enterprise data: Agent Framework, Agent Evaluation, Model Training and Gateway in public preview, Tools Catalog in private preview, MLflow 2.14 with tracing.

Enterprise AI DatabricksMosaic AIAgent Framework

June 10, 2024 High

Apple Intelligence: Apple's AI plan, on-device + Private Cloud Compute

At WWDC on June 10, 2024 Apple unveils Apple Intelligence: on-device models on iPhone 15 Pro and M1-or-later devices, heavier requests on Private Cloud Compute, optional ChatGPT (GPT-4o) integration. Beta in the fall, US English only.

Enterprise AI AppleApple IntelligenceWWDC

June 6, 2024 High

Kling AI (Kuaishou): videos up to 2 minutes at 1080p, in public beta in China

Kuaishou unveils Kling, a diffusion-transformer text-to-video model claiming clips up to 2 minutes, 1080p and 30 fps. The beta opens to Chinese users of the KuaiYing app, while Sora stays closed.

Image & Video Gen Kling AIVideo GenerationKuaishou

June 6, 2024 Medium

Tabby v0.12: the self-hosted coding assistant connects to Ollama and GitLab

TabbyML releases Tabby v0.12.0, a self-hosted AI coding assistant: GitLab SSO login, connection to self-hosted GitHub and GitLab, repository context in the Code Browser too, and a non-experimental HTTP API to use models via llama.cpp, Ollama, Mistral/Codestral and OpenAI.

Local AI TabbyMLTabbycoding assistant

June 5, 2024 Medium

Stable Audio Open: open weights for sound effects and audio samples

Stability AI releases on Hugging Face the weights of Stable Audio Open, a roughly 1-billion-parameter text-to-audio model that generates up to 47 seconds of 44.1 kHz stereo audio, trained on Freesound and Free Music Archive recordings.

Voice & Audio Stable Audiosound effectsopen weights

June 3, 2024 Low

Jan 0.5.0: refreshed desktop interface for the offline LLM app

Jan, the open-source app for running language models locally, releases version 0.5.0 with a redesigned desktop interface, Hugging Face access token registration and a Model Hub update.

Local AI Jan.aiDesktop AppOffline

June 2, 2024 High

NVIDIA NIM inference microservices become available to developers

At Computex 2024 NVIDIA makes NIM available: models packaged as optimized inference containers that developers can download and run on cloud, data center or workstation GPUs, with production use through NVIDIA AI Enterprise.

AI Infrastructure NVIDIANIMInference

June 1, 2024 Medium

Open WebUI Pipelines: an external plugin framework for the local LLM frontend

With v0.2.0 (June 1, 2024) Open WebUI introduces Pipelines: a separate, OpenAI API-compatible Python server that adds custom logic (function calling, per-user rate limits, Langfuse monitoring, toxic message filters) without modifying the core.

Local AI Open WebUIPipelinesplugin

May 2024

May 31, 2024 Medium

Cartesia Sonic: Cartesia's first voice model, built on state space models, at 135 ms

At the end of May 2024 the startup Cartesia unveiled Sonic, a text-to-speech model built on a state space model architecture rather than transformers, with 135 ms model latency, a web playground, a low-latency API, instant voice cloning and voice design.

Voice & Audio CartesiaSonicTTS

May 29, 2024 High

Codestral: Mistral's first code model, 22B parameters and 80+ languages

On May 29, 2024 Mistral launches Codestral, a 22B-parameter open-weight model dedicated to code: trained on more than 80 languages, 32k-token context, fill-in-the-middle, Mistral AI Non-Production License.

AI Coding Code LLMOpen WeightsVS Code

May 25, 2024 Medium

Promptfoo 0.60.0: automatic red teaming test generation arrives

With version 0.60.0 the open source Promptfoo framework, built to evaluate prompts and LLM applications, adds automatic generation of red teaming tests against harmful outputs, jailbreaks and app hijacking.

AI Security PromptfooRed TeamingOpen Source

May 22, 2024 High

Aider: terminal coding agent, state of the art on SWE-bench Lite with 26.3%

On May 22, 2024 Aider, an open source terminal coding assistant with automatic git commits, solves 26.3% of SWE-bench Lite by alternating GPT-4o and Claude 3 Opus, above the 20.3% of Amazon Q Developer Agent. On June 2 it reaches 18.9% on full SWE-bench, versus Devin's 13.9% on the same 570 cases.

AI Coding AiderCoding AgentCLI

May 21, 2024 Medium

Phi-3-vision (Microsoft): the first multimodal model in the Phi-3 family, 4.2B parameters

At Build 2024 Microsoft introduces Phi-3-vision: 4.2 billion parameters, 128K-token context, built on Phi-3-mini; reads text in images and interprets charts, diagrams and tables. Open weights under the MIT license.

Multimodal AI Phi-3 VisionMicrosoftsmall VLM

May 21, 2024 Medium

GitHub Copilot Extensions: third-party tools inside Copilot Chat

GitHub announces Copilot Extensions in limited public beta: tools from partners such as DataStax, Docker, LaunchDarkly, Microsoft Azure, MongoDB, Sentry and Stripe usable from Copilot Chat, plus private extensions for organizations.

AI Coding GitHub Copilotextensionsmarketplace

May 21, 2024 High

Mapping the Mind: Anthropic extracts millions of interpretable features from Claude 3 Sonnet

On 21 May 2024 Anthropic published interpretability research in which, by applying dictionary learning to a middle layer of Claude 3 Sonnet, it identifies millions of features matching concepts such as the Golden Gate Bridge, code bugs or sycophancy, and shows that amplifying them changes the model's behavior.

AI Security InterpretabilityAnthropicClaude 3 Sonnet

May 20, 2024 High

Copilot+ PC and Recall: Microsoft tries a PC 'memory', privacy backlash erupts

Microsoft unveils Copilot+ PCs with 40+ TOPS NPUs and the Recall feature, which saves screen snapshots into a searchable local index. Researchers show the database is readable in plain text: on June 13 Recall is pulled from the June 18 launch and limited to Insiders.

AI Security MicrosoftCopilot+ PCRecall

May 13, 2024 High

GPT-4o: text, voice and images in a single model

OpenAI unveils GPT-4o (omni), a single model that natively handles text, audio, and images with 320 ms average voice response, GPT-4 Turbo-level text quality, and text+image capabilities opened to free users too.

Multimodal AI OpenAIGPT-4oVoice

May 13, 2024 Medium

Unitree G1: a $16,000 humanoid

Unitree unveils the G1, a humanoid about 1.3 meters tall weighing about 35 kg, starting at $16,000: 23 to 43 degrees of freedom, optional Dex3-1 hand on the EDU version, trained with imitation and reinforcement learning according to the company.

Robotics UnitreeG1Humanoid

May 8, 2024 Landmark

AlphaFold 3: from protein structure to all of life's molecular interactions

Google DeepMind and Isomorphic Labs publish AlphaFold 3 in Nature: it predicts the structure and interactions of complexes of proteins, DNA, RNA, ligands and ions, extending the domain of AlphaFold 2.

Foundation Models DeepMindAlphaFoldBiology

May 8, 2024 Medium

Msty: a desktop app to use and compare local and online models

Msty is a free (proprietary) desktop app for Mac, Windows and Linux that brings local models, for example via Ollama, and online models into one interface, with split chats to compare answers.

Local AI MstyGUIMulti-model

May 6, 2024 High

DeepSeek-V2: a 236B open MoE with Multi-head Latent Attention and an API at 1 yuan per million tokens

DeepSeek releases V2: 236B-total / 21B-active MoE, 128K context, with Multi-head Latent Attention (MLA) cutting the KV cache by 93.3% versus DeepSeek 67B. The API costs 1 yuan per million input tokens, in the same weeks a price war on LLMs starts in China.

Open Source Models DeepSeekMoEMLA

May 6, 2024 High

Hugging Face LeRobot: an open-source library for robot learning

In May 2024 Hugging Face launches LeRobot, an open-source PyTorch library for robotics led by Remi Cadene (ex Tesla): models, datasets, and tools shared on the Hub, with pretrained policies and simulation environments.

Robotics HuggingFaceLeRobotOpen Source

May 3, 2024 Medium

HuggingFace Accelerate 0.30: FSDP vs DeepSpeed guide, MoE with DeepSpeed and new optimizers

Accelerate 0.30.0 adds MoE model support with DeepSpeed, upcasting for FSDP in mixed precision, the schedule-free and LOMO optimizers, and a new concept guide explaining how FSDP and DeepSpeed map to each other.

AI Infrastructure HuggingFaceAccelerateFSDP

May 3, 2024 Medium

DrEureka: GPT-4 designs rewards and domain randomization to take robots from simulation to the real world

University of Pennsylvania, NVIDIA and UT Austin present DrEureka: GPT-4 writes the reward function and picks domain randomization parameters for sim-to-real transfer. Tested on a Unitree Go1 quadruped and a LEAP hand, it reaches results competitive with hand-designed configurations.

Robotics DrEurekasim-to-realdomain randomization

May 1, 2024 High

Atlassian Rovo: AI search, chat and agents on company data, from Jira to Google Drive

At Team '24 Atlassian launches Rovo: AI search across Jira, Confluence and external tools (Google Drive, SharePoint, Teams, GitHub, Slack, Figma) respecting permissions, a chat that cites sources, and agents that can be built in natural language.

Enterprise AI AtlassianRovoJira

May 1, 2024 Medium

Anthropic launches the Claude Team plan and the iOS app

Anthropic introduces the Team plan at $30 per user per month (5-seat minimum): more usage than Pro, access to Claude 3 Opus, Sonnet and Haiku, a 200K-token context window and admin tools. The Claude iOS app ships alongside it.

Enterprise AI AnthropicClaude Teamenterprise

April 2024

April 30, 2024 Medium

LLaVA-NeXT-Video: an image-trained model that also understands video

On April 30, 2024 the LLaVA team (ByteDance, UW-Madison, NTU) publishes LLaVA-NeXT-Video: LLaVA-NeXT, trained only on images, shows zero-shot video understanding; with video fine-tuning and DPO from LLM feedback, the 7B and 34B models beat the video-specific open LMMs they are compared with.

Multimodal AI LLaVA-NeXTVideo UnderstandingZero-Shot

April 30, 2024 Medium

Amazon Q Developer goes GA: /dev and /transform agents in the IDE

On April 30, 2024 AWS makes Amazon Q Developer generally available. The IDE gets two agents: /dev, which proposes an implementation plan and then the code, and /transform, which upgrades Java 8 and 11 projects to Java 17. Free and Pro tiers ($19 per user per month).

AI Coding Amazon QAmazon Q Developercoding agent

April 24, 2024 Medium

Snowflake Arctic: 480B-total / 17B-active MoE under Apache 2.0

Snowflake releases Arctic, a hybrid Dense-MoE model with 480B total and 17B active parameters, geared toward SQL, coding and instruction following, under Apache 2.0 and with a stated training cost below 2 million dollars.

Open Source Models SnowflakeArcticMoE

April 23, 2024 High

Phi-3: Microsoft relaunches SLMs, a 3.8B model that runs on iPhone

Microsoft releases Phi-3-mini (3.8B, 4K and 128K context) and announces Phi-3-small (7B) and Phi-3-medium (14B). According to the technical report mini is on par with Mixtral 8x7B and GPT-3.5 on several benchmarks and runs offline on an iPhone 14.

Local AI MicrosoftPhi-3Small Language Models

April 19, 2024 Medium

CyberSecEval 2: Meta's benchmark for LLM cybersecurity risks

Meta publishes CyberSecEval 2, the second version of Purple Llama's open-source suite for evaluating LLM security risks. It adds tests for prompt injection, code interpreter abuse and exploit capabilities, plus a false refusal measure. All tested models fall for prompt injection in 26-41% of cases.

AI Security CyberSecEvalMetaPurple Llama

April 18, 2024 High

Llama 3: Meta releases open 8B and 70B models

On April 18, 2024 Meta releases Llama 3 in two sizes (8B and 70B), base and Instruct: over 15T training tokens, 128K tokenizer, 8K context. The 70B Instruct scores 82.0 on MMLU and 81.7 on HumanEval; models above 400B are still training.

Open Source Models MetaLlama 3Open Weights

April 17, 2024 High

Boston Dynamics electric Atlas: hydraulics retired, industrial robot born

On April 16, 2024 Boston Dynamics retires the hydraulic Atlas; the next day it unveils the new fully electric Atlas, designed for real-world applications, with Hyundai as the first testing partner.

Robotics Boston DynamicsAtlasHumanoid Robot

April 17, 2024 High

Mixtral 8x22B: Mistral's Apache 2.0 MoE with 39B active parameters

On April 17, 2024 Mistral officially presents Mixtral 8x22B, a sparse MoE with 141B total and 39B active parameters per token, 64k context, native function calling and an Apache 2.0 license, together with the Instruct version.

Open Source Models MistralMixtralMoE

April 16, 2024 Medium

Zendesk Relate 2024: autonomous AI agents and copilot for customer service

At its Relate conference on April 16, 2024 Zendesk unveils autonomous AI agents that aim to resolve customer requests end-to-end, an agent copilot for human staff, and new Workforce Management and Quality Assurance features.

Enterprise AI ZendeskCustomer SupportAI Agents

April 15, 2024 High

IDEFICS2: 8B open multimodal model with Apache 2.0 weights

On April 15, 2024 Hugging Face releases Idefics2, 8B parameters, Apache 2.0 weights, images up to 980x980 at native aspect ratio and OCR data in pre-training to read text in images and documents better.

Multimodal AI IDEFICS2HuggingFaceOCR

April 10, 2024 High

Udio: full songs with vocals generated from a prompt, free public beta

On April 10, 2024 Udio, founded by former Google DeepMind researchers and backed by a16z, opens the free public beta of its app: it generates tracks with vocals and lyrics from a text description.

Voice & Audio UdioMusic GenerationAI Music

April 4, 2024 Medium

Cohere Command R+: an enterprise-focused model built for RAG and tool use

Cohere launches Command R+, a 104B-parameter model with 128k context optimized for RAG and multi-step tool use: it debuts on Microsoft Azure and the weights are on Hugging Face under the non-commercial CC-BY-NC 4.0 license.

Enterprise AI CohereCommand R+RAG

April 3, 2024 Medium

Stable Audio 2.0: stereo tracks up to 3 minutes and audio-to-audio

Stability AI launches Stable Audio 2.0: 44.1 kHz stereo tracks up to three minutes with coherent musical structure, prompt-driven transformation of uploaded audio, and a diffusion-transformer architecture.

Voice & Audio Stability AIStable AudioMusic Generation

April 3, 2024 Medium

Plandex: an open source terminal AI coding engine for complex multi-file tasks

In a Show HN post Plandex presents itself as an open source, terminal-based AI coding engine for multi-file, multi-step tasks: changes accumulate in a version-controlled sandbox and can be reviewed, rejected or rewound before being applied.

AI Coding PlandexCoding AgentTerminal

April 2, 2024 Medium

Crescendo: the multi-turn jailbreak that bypasses guardrails through gradual escalation

Mark Russinovich, Ahmed Salem and Ronen Eldan of Microsoft publish Crescendo, a multi-turn jailbreak that starts with innocuous questions and leverages the model's own answers; successfully tested on ChatGPT, Gemini Pro, Gemini Ultra, LLaMA-2 70b Chat and Anthropic Chat.

AI Security JailbreakMulti-TurnMicrosoft

April 2, 2024 High

SWE-agent: Princeton's open source agent solving 12% of SWE-bench issues

On April 2, 2024 Princeton's NLP group releases SWE-agent, an open source agent that with GPT-4 Turbo solves 12.29% of real issues across the full SWE-bench thanks to an agent-computer interface (ACI) designed for language models. The May paper reports 12.5%.

Agents PrincetonSWE-agentSWE-bench

April 2, 2024 High

Many-Shot Jailbreaking: safety training overridden by context length

Anthropic publishes research on many-shot jailbreaking: placing up to 256 fake dialogues in which the assistant answers harmful requests into the prompt makes the model more likely to do the same. Effectiveness follows a power law in the number of examples; Anthropic briefed other AI developers in advance.

AI Security many-shotjailbreakinglong context

March 2024

March 30, 2024 High

Automatic Prefix Caching in vLLM 0.4.0: KV cache reused across requests with the same prefix

vLLM 0.4.0 (March 30, 2024) ships Automatic Prefix Caching, merged on March 2 via PR #2762: KV cache blocks of identical prefixes, such as a long system prompt, are reused across different requests. It is enabled with --enable-prefix-caching.

AI Infrastructure prefix cachingKV cachevLLM

March 27, 2024 Medium

DBRX: Databricks's 132B-total / 36B-active open MoE

Databricks releases DBRX, an open-weights Mixture-of-Experts with 132B total and 36B active parameters, 16 experts with 4 chosen per token; DBRX Instruct scores 73.7% on MMLU and 70.1% on HumanEval.

Open Source Models DatabricksDBRXMoE

March 21, 2024 Medium

Devika: an open-source alternative to Devin

Mufeed VH (Stition.AI, Kerala) publishes Devika, an MIT-licensed open-source AI software engineer agent: it takes high-level instructions, breaks them into steps, researches the web and writes code. It is a response to Cognition AI's Devin.

Agents Devikaopen sourcesoftware engineer agent

March 21, 2024 Medium

Suno v3: full two-minute songs, better audio and more genres

Suno releases v3, which generates full two-minute songs in seconds, with better audio quality, more styles and genres, and improved prompt adherence with fewer hallucinations and more graceful endings.

Voice & Audio SunoMusic GenerationAI Music

March 19, 2024 High

DROID: 76,000 robot manipulation demonstrations collected across 564 scenes

A consortium including Stanford, UC Berkeley and Toyota Research Institute releases DROID: about 76,000 trajectories (about 350 hours) across 564 scenes and 86 tasks, collected at 13 institutions with the same Franka Panda setup. Dataset released under CC BY 4.0.

Robotics DROIDrobot datasetmanipulation

March 18, 2024 Landmark

NVIDIA Blackwell: B200 and GB200 NVL72, the rack-scale AI era

At GTC 2024 NVIDIA announces Blackwell (208 billion transistors, two dies acting as one GPU) and the GB200 NVL72 rack with 72 GPUs and 36 Grace CPUs. Claimed up to 30x LLM inference versus the same number of H100s.

AI Infrastructure NVIDIABlackwellB200

March 13, 2024 Landmark

EU AI Act: European Parliament adopts the first comprehensive AI law

On March 13, 2024 the European Parliament approves the AI Act (523 in favour, 46 against, 49 abstentions), the first comprehensive law on artificial intelligence, with a risk-based approach and specific obligations for general-purpose models. Lawyer-linguist review and Council endorsement are still pending.

AI Security EU AI ActRegulationEurope

March 13, 2024 High

Figure 01 + OpenAI: the humanoid that talks, reasons and picks up objects

Figure publishes a video of its Figure 01 humanoid conversing with an engineer and manipulating objects: a multimodal model trained by OpenAI handles dialogue and behavior selection, Figure's neural networks execute the movements.

Robotics FigureOpenAIHumanoid

March 13, 2024 Medium

Cerebras unveils WSE-3 and the CS-3 system: 4 trillion transistors on a wafer-sized chip

On 13 March 2024 Cerebras announces its third-generation Wafer-Scale Engine (WSE-3) and the CS-3 system: 4 trillion transistors, 900,000 AI cores, 44 GB of on-chip SRAM, 125 petaflops of peak AI performance, TSMC 5 nm process. The same day G42 and Cerebras start Condor Galaxy 3, a 64-CS-3, 8-exaflop supercomputer.

AI Infrastructure CerebrasWafer-ScaleWSE-3

March 12, 2024 High

Devin: Cognition unveils its autonomous 'AI software engineer'

Cognition announces Devin, an agent that plans and executes development tasks with a shell, editor and browser in a sandbox. It claims 13.86% on SWE-bench, measured on a random 25% subset (570 of 2,294 tasks).

Agents CognitionDevinAutonomous Agent

March 8, 2024 Medium

bitsandbytes 0.43: official QLoRA + FSDP support and Windows

On March 8, 2024 bitsandbytes 0.43.0 makes QLoRA with FSDP officially supported, enabling fine-tuning of roughly 70B models across several 24 GB consumer GPUs, and adds official Windows support via pip.

AI Infrastructure bitsandbytesQLoRAFSDP

March 5, 2024 Medium

Box AI generally available to all Enterprise Plus customers, with Azure OpenAI Service

Box makes Box AI generally available on Enterprise Plus plans, together with a Microsoft Azure OpenAI Service integration: 20 queries per month per user and 2,000 additional at company level.

Enterprise AI BoxBox AIDocument AI

March 5, 2024 High

Stable Diffusion 3: the paper reveals the MMDiT architecture and rectified flow

On March 5, 2024 Stability AI publishes the Stable Diffusion 3 paper (announced on February 22): MMDiT transformer with separate weights for text and image, rectified flow training, variants from 800M to 8B parameters.

Image & Video Gen Stability AIStable Diffusion 3MMDiT

March 4, 2024 Landmark

Claude 3 (Opus, Sonnet, Haiku): Anthropic challenges GPT-4

Anthropic ships the Claude 3 family in three sizes, all with vision and a 200K-token context. According to Anthropic, Opus outperforms its peers on most common benchmarks, GPT-4 included. Opus and Sonnet available immediately, Haiku coming soon.

Foundation Models AnthropicClaude 3Opus

March 1, 2024 Medium

Unitree H1 V3.0 Evolution: speed record for a full-size humanoid

Unitree shows its H1 humanoid walking at 3.3 m/s on flat ground, beating the 2.5 m/s of Boston Dynamics' Atlas; expected price between 90,000 and 150,000 dollars.

Robotics UnitreeH1Humanoid Robot

February 2024

February 28, 2024 Medium

StarCoder2 and The Stack v2: open code models at 3B, 7B and 15B

BigCode releases StarCoder2 in three sizes (3B, 7B, 15B), trained on 3.3-4.3 trillion tokens from The Stack v2, built with Software Heritage on repositories spanning 619 languages. The 15B matches or beats CodeLlama-34B. BigCode OpenRAIL-M license.

AI Coding StarCoder2BigCodeThe Stack v2

February 28, 2024 Medium

Ideogram 1.0: the image generator built around legible text

On February 28, 2024 Ideogram introduces its 1.0 model together with an $80 million Series A led by Andreessen Horowitz. The company claims about half DALL-E 3's text rendering error rate and introduces Magic Prompt.

Image & Video Gen IdeogramText RenderingText-to-Image

February 28, 2024 Medium

NeMo Guardrails 0.8: Colang 2.0 preview and jailbreak heuristics

NVIDIA releases NeMo Guardrails 0.8.0, the open source toolkit for adding programmable guardrails to LLM applications: Colang 2.0 arrives as a preview, along with jailbreak-detection heuristics, generation options, configurable stop tokens and embedding caching.

AI Security NVIDIANeMo GuardrailsOpen Source

February 26, 2024 Medium

Mistral Large and Le Chat: Mistral's flagship arrives via API and on Azure

Mistral AI announces Mistral Large, a flagship model available via API that the company ranks second only to GPT-4 on several benchmarks, together with Mistral Small, the Le Chat assistant in beta, and distribution on Microsoft Azure.

Foundation Models MistralMistral LargeLe Chat

February 26, 2024 Medium

NextChat v2.11.2: local models via Ollama in the self-hosted UI

NextChat (formerly ChatGPT-Next-Web), an MIT-licensed open-source ChatGPT-style web interface, ships v2.11.2 with support for local models such as Gemma, Mistral and Llama through Ollama.

Local AI NextChatChatGPT-Next-Webself-hosted

February 21, 2024 High

Gemini for Google Workspace: Duet AI is renamed, Gemini Business and Enterprise arrive

On February 21, 2024 Google renames Duet AI for Google Workspace to Gemini for Google Workspace and launches two paid add-ons: Gemini Business at 20 dollars and Gemini Enterprise at 30 dollars per user per month with an annual commitment, plus a standalone Gemini chat with enterprise data protections.

Enterprise AI Google GeminiWorkspaceGemini Business

February 21, 2024 High

Gemma: Google enters the open-weights game

Google releases Gemma 2B and 7B, open-weight models built from Gemini research and technology, in pretrained and instruction-tuned versions, with terms of use that allow commercial use.

Open Source Models GoogleGemmaOpen Weights

February 19, 2024 Medium

MeloTTS: open-source multilingual speech synthesis, fast even on CPU

MyShell.ai and MIT publish MeloTTS, an MIT-licensed multilingual text-to-speech library: English (five accents), Spanish, French, Chinese with mixed English, Japanese and Korean. According to the authors it is fast enough for real-time inference on CPU.

Voice & Audio MeloTTSTTSmultilingual

February 19, 2024 High

Groq LPU: 500-tokens-per-second inference goes viral

In mid-February 2024 Groq's public demo on open models such as Mixtral 8x7B generates about 500 tokens per second and goes viral on X; GroqCloud launches for developers in the same days.

AI Infrastructure GroqLPUInference

February 17, 2024 High

Ollama WebUI becomes Open WebUI: self-hosted, multi-user ChatGPT-style interface

On 17 February 2024 the Ollama WebUI project, started in October 2023, is renamed Open WebUI: a self-hosted web interface for Ollama and OpenAI-compatible APIs, with document RAG, user roles and Docker installation.

Local AI Open WebUIOllamaChatGPT UI

February 15, 2024 High

Gemini on Vertex AI: 1.0 Pro available to all, 1.0 Ultra via allowlist, 1.5 Pro in preview

On February 15, 2024 Google Cloud makes Gemini 1.0 Pro generally available on Vertex AI, opens Gemini 1.0 Ultra to allowlisted customers and puts Gemini 1.5 Pro in private preview, with LoRA tuning, grounding and function calling.

Enterprise AI GoogleVertex AIGemini

February 15, 2024 High

Gemini 1.5 Pro: 1 million tokens in context

Google announces Gemini 1.5 Pro: Mixture of Experts architecture, 128K standard context and 1 million in private preview, quality comparable to Gemini 1.0 Ultra with less compute.

Foundation Models GoogleGemini 1.5Long Context

February 15, 2024 Landmark

Sora: OpenAI shows minute-long AI video from a prompt

OpenAI announces Sora, a text-to-video model that generates clips up to one minute long while keeping visual quality and prompt adherence. Initial access only for red teamers and a group of visual artists, designers and filmmakers.

Image & Video Gen OpenAISoraText-to-Video

February 13, 2024 Medium

ChatGPT Memory: OpenAI starts testing cross-conversation memory

OpenAI announces a test of Memory in ChatGPT with a small portion of Free and Plus users: the model can remember information across chats, with controls to make it forget or turn the feature off.

Foundation Models OpenAIChatGPTMemory

February 12, 2024 Medium

LlamaIndex v0.10: the RAG framework splits into core and integration packages

LlamaIndex releases v0.10: llama-index-core is born, each integration becomes a separate PyPI package, LlamaHub becomes the single hub for integrations and ServiceContext is deprecated in favor of Settings. The team warns the upgrade may break existing code.

AI Infrastructure LlamaIndexRAGllama-index-core

February 8, 2024 High

Ollama becomes OpenAI API compatible: local models speak Chat Completions

On February 8, 2024 Ollama adds experimental support for OpenAI's Chat Completions API at /v1/chat/completions: the OpenAI Python and JavaScript SDKs work with local models by changing only the base_url.

Local AI OllamaOpenAI CompatibleChat Completions

February 8, 2024 Medium

UFO: Microsoft's agent that operates Windows applications via GPT-4V and UI Automation

Microsoft publishes UFO (UI-Focused Agent), a dual-agent framework that observes Windows application screenshots and controls with GPT-Vision and acts through the UI Automation API. On 50 requests across 9 applications it reaches an 86% success rate. Open-source code under the MIT license.

Agents UFOWindows agentUI Automation

February 7, 2024 Medium

ALOHA 2: Google DeepMind and Stanford improve the open hardware for bimanual teleoperation

On 7 February 2024 a Google DeepMind and Stanford team published ALOHA 2, an improved version of the ALOHA system for collecting two-arm manipulation demonstrations: low-friction grippers, new gravity compensation, a simplified frame, RealSense cameras. Open-source hardware designs and a MuJoCo model included.

Robotics Google DeepMindStanfordALOHA 2

February 6, 2024 Medium

Stable Video Diffusion 1.1: video from a single image, more consistent

Stability AI releases SVD 1.1, a fine-tune of SVD-XT that generates 25-frame clips at 1024x576 from a single image, with weights downloadable from Hugging Face.

Image & Video Gen Stability AISVDVideo Generation

February 6, 2024 High

HarmBench: a standardized benchmark for automated red teaming and LLM defenses

Mazeika, Hendrycks and colleagues (Center for AI Safety, UIUC, CMU, UC Berkeley, Microsoft) release HarmBench: 510 harmful behaviors, 18 red teaming methods compared across 33 LLMs and defenses. No attack or defense is uniformly effective. The paper also proposes R2D2, an adversarial training method.

AI Security HarmBenchjailbreakevaluation

February 4, 2024 High

Qwen1.5: six sizes from 0.5B to 72B, 32K context across the board

Alibaba's Qwen team releases Qwen1.5 in six sizes (0.5B, 1.8B, 4B, 7B, 14B, 72B), base and chat, with 32K-token context on every model, quantized versions and direct integration into Hugging Face transformers.

Foundation Models QwenAlibabaMultilingual

February 1, 2024 High

OLMo: AllenAI releases an open LLM with weights, data, code and checkpoints

AllenAI releases OLMo (1B and 7B) together with the Dolma training corpus, training and evaluation code, and more than 500 intermediate checkpoints per model, all under Apache 2.0, to enable scientific study of LLM training.

Foundation Models OLMoAllenAIopen source

January 2024

January 31, 2024 Medium

vLLM 0.3.0: FP8 (E5M2) KV cache arrives

On January 31, 2024 vLLM 0.3.0 adds support for an FP8 E5M2 quantized KV cache: the attention cache uses less GPU memory and throughput can increase. It is enabled with kv_cache_dtype="fp8".

AI Infrastructure KV cache quantizationFP8E5M2

January 29, 2024 Medium

Code Llama 70B: Meta adds the largest size to the Code Llama family

Meta releases Code Llama 70B in three variants (base, Python, Instruct), trained on 1T tokens; the Instruct version scores 67.8 on HumanEval.

AI Coding MetaCode LlamaOpen Source

January 28, 2024 Medium

llama.cpp gets a Vulkan backend: GPU acceleration beyond CUDA and Metal

On January 28, 2024 the Vulkan backend written by 0cc4m lands in llama.cpp, meant to supersede OpenCL as the widely compatible GPU backend. The next day Nomic's Kompute backend, another Vulkan-based implementation, is merged too. It is a first landing, with much optimization still to do.

Local AI llama.cppVulkanKompute

January 20, 2024 Medium

moondream1: a 1.6B VLM that runs on CPU

On January 20, 2024 independent developer vikhyatk releases moondream1, a 1.6-billion-parameter vision-language model built with SigLIP, Phi-1.5 and the LLaVA dataset. Research-only license, commercial use prohibited.

Multimodal AI VLMTiny ModelSigLIP

January 18, 2024 High

DistServe: prefill and decode on separate GPUs for LLM serving

On January 18, 2024 DistServe appears on arXiv (later OSDI 2024): prefill and decode run on different GPUs to remove interference between the two phases. Against the systems of the time it claims up to 7.4x more requests served or 12.6x tighter SLOs. It follows Splitwise from Microsoft and the University of Washington by seven weeks.

AI Infrastructure PrefillDecodeDisaggregazione

January 17, 2024 High

LangGraph: stateful agents as cyclic graphs with loops and branching

LangChain introduces LangGraph, a module built on top of LangChain to create agent runtimes as graphs with cycles: a StateGraph with shared state, nodes, conditional edges and an END exit point.

Agents LangGraphLangChainStateful Agents

January 17, 2024 High

AlphaGeometry: DeepMind solves olympiad-level geometry

Google DeepMind publishes AlphaGeometry in Nature, a neuro-symbolic system that solves 25 of 30 IMO geometry problems on the IMO-AG-30 benchmark, close to the average gold medalist's 25.9, trained without human demonstrations.

Foundation Models DeepMindAlphaGeometryReasoning

January 17, 2024 Medium

SGLang: RadixAttention and a structured language, up to 5x throughput

LMSYS introduces SGLang, a language and runtime for structured LLM programs: RadixAttention automatically reuses the KV cache across different calls, with up to 5x higher throughput than Guidance and vLLM in the blog's tests.

AI Infrastructure SGLangLMSYSRadixAttention

January 10, 2024 Medium

GPT Store: the custom GPTs marketplace opens

On January 10, 2024 OpenAI opens the GPT Store inside ChatGPT for Plus, Team and Enterprise subscribers. According to OpenAI, users had already created over 3 million GPTs; a creator payout program for US builders is announced for the first quarter.

Enterprise AI OpenAIGPT StoreGPTs

January 10, 2024 High

Sleeper Agents (Anthropic): backdoored models survive safety training

Hubinger et al. (Anthropic) train LLMs with deliberate backdoors and show that supervised fine-tuning, reinforcement learning and adversarial training do not remove them; persistence is highest in the largest models and in those trained to reason in chain-of-thought about deceiving training.

AI Security Sleeper AgentsAnthropicBackdoor

January 4, 2024 High

Mobile ALOHA: low-cost bimanual mobile manipulation from Stanford

Zipeng Fu, Tony Z. Zhao and Chelsea Finn (Stanford) add a mobile base and whole-body teleoperation to the ALOHA system. With 50 demonstrations per task and co-training on static ALOHA data, the robot autonomously performs tasks such as cooking a shrimp, calling an elevator and rinsing a pan.

Robotics Mobile ALOHAbimanualmobile robot

2023

December 2023

December 27, 2023 Medium

AnythingLLM adds Ollama: fully local document chat

On December 27, 2023 AnythingLLM, Mintplex Labs' open source (MIT) app for chatting with your own documents, adds Ollama as a model provider. With LocalAI, LM Studio, the native embedder and built-in LanceDB, the RAG pipeline can run without cloud services.

Local AI AnythingLLMRAG LocaleVector DB

December 21, 2023 High

InternVL: a 6-billion-parameter vision encoder aligned with LLMs

The InternVL paper (arXiv, 21 December 2023) scales the vision foundation model to 6B parameters with InternViT-6B and progressively aligns it with an LLM, claiming state-of-the-art results on 32 visual-linguistic benchmarks.

Multimodal AI InternVLOpen SourceVisual Encoder

December 18, 2023 High

OpenAI Preparedness Framework (beta): risk thresholds before development and release

OpenAI publishes the beta Preparedness Framework: four risk categories (cybersecurity, CBRN, persuasion, model autonomy) with levels from low to critical and rules on what can be released or developed further.

AI Security OpenAIPreparedness FrameworkFrontier AI

December 15, 2023 High

AMD ROCm 6.0: the software stack for Instinct MI300 and generative AI

Announced on 6 December 2023 with the Instinct MI300X and MI300A and released on 15 December, ROCm 6.0 enables the MI300 series, adds FlashAttention, HIPGraph and vLLM, and upstream support for PyTorch, TensorFlow and JAX.

AI Infrastructure ROCm 6AMDMI300X

December 14, 2023 Medium

Solar 10.7B: Upstage depth-scales a 7B model and tops the Open LLM Leaderboard

Upstage releases Solar 10.7B, built with depth up-scaling from Mistral 7B weights: on 14 December 2023 it announces first place on the Hugging Face Open LLM Leaderboard with an average of 74.2. The base model is licensed under Apache 2.0.

Foundation Models SolarUpstageDepth Up-Scaling

December 14, 2023 Medium

StyleTTS 2 at NeurIPS 2023: human-level speech synthesis with style diffusion

On December 14, 2023 StyleTTS 2 (Columbia University) is presented as a poster at NeurIPS 2023. It uses style diffusion and WavLM-based discriminators: on LJSpeech listeners prefer it to the original recordings, on VCTK they rate it on par.

Voice & Audio StyleTTS2TTSStyle Diffusion

December 13, 2023 High

Tesla Optimus Gen 2: 30% faster, 10 kg lighter, hands with tactile sensors

Tesla publishes a video of the Optimus Gen 2 prototype: 30% faster walking, 10 kg less weight, 11-degree-of-freedom hands with tactile sensing on all fingers, shown handling an egg.

Robotics TeslaOptimusHumanoid Robot

December 12, 2023 Medium

Phi-2: Microsoft's 2.7B model that beats Llama-2 13B

On December 12, 2023 Microsoft Research introduces Phi-2, a 2.7-billion-parameter base model trained on 1.4T tokens of synthetic and filtered web data. In the blog's benchmarks it beats Llama-2 7B and 13B and, in several test groups, Mistral 7B.

Local AI MicrosoftPhi-2SLM

December 11, 2023 Landmark

Mixtral 8x7B: open-weights Mixture of Experts at GPT-3.5 level

After a magnet link posted on X on December 8 with no explanation, on December 11 Mistral announces Mixtral 8x7B: Sparse MoE with 8 experts, 46.7B total parameters and 12.9B active per token, 32k context. Matches or beats GPT-3.5 and Llama 2 70B on most benchmarks, Apache 2.0 license.

Open Source Models MistralMixtralMoE

December 7, 2023 High

Llama Guard: an LLM trained to be the gatekeeper of other LLMs

Meta publishes Llama Guard, a safety classifier based on Llama 2-7B that evaluates both user prompts and model responses against a 6-category risk taxonomy. It ships with open weights as part of the Purple Llama project.

AI Security MetaLlama GuardPurple Llama

December 6, 2023 Medium

JetBrains AI Assistant leaves preview in its 2023.3 IDEs

On 6 December 2023 JetBrains makes AI Assistant available in the 2023.3 versions of all its commercial IDEs, ReSharper and Fleet: project-aware chat, suggested refactoring, completion, documentation and commit messages. A separate JetBrains AI subscription is required.

AI Coding JetBrainsAI AssistantIntelliJ

December 6, 2023 Medium

Jan 0.4.0: open source desktop app for local LLMs, with a model hub and a server on port 1337

On 6 December 2023 Jan v0.4.0 ships, an open source desktop app (AGPLv3, built on Electron) for running LLMs locally: it revamps the thread and model hub screens and adds the groundwork for a local server on localhost:1337. The project still describes itself as in development.

Local AI JanDesktop AppOpen Source

December 6, 2023 Landmark

Google Gemini 1.0: natively multimodal in three sizes

Google announces Gemini 1.0 in three sizes (Ultra, Pro, Nano), trained from the start on text, images, audio and video. Ultra claims 90.0% on MMLU and state of the art on 30 of 32 benchmarks. Pro ships in Bard right away; the demo video turns out to be edited.

Foundation Models GoogleGeminimultimodal

December 5, 2023 High

MLX: Apple's ML research team releases an open-source framework for Apple Silicon

On 5 December 2023 Apple's machine learning research group publishes MLX, a NumPy-style array framework designed for Apple Silicon, with unified memory, lazy computation and an MIT license.

Local AI MLXApple SiliconUnified Memory

November 2023

November 30, 2023 Medium

Microsoft TaskWeaver: an agent framework that thinks in code

Microsoft publishes TaskWeaver on GitHub, a code-first framework for LLM agents aimed at data analytics: requests become executable code, plugins are callable functions, and execution state persists across steps.

Agents TaskWeaverMicrosoftCode-First

November 29, 2023 Medium

Mozilla llamafile: LLM in a single portable executable on any OS

Mozilla introduces llamafile, a project combining llama.cpp with Cosmopolitan Libc to turn LLM weights into a single executable that runs on six operating systems without installation.

AI Infrastructure llamafileMozillaLLM

November 28, 2023 Medium

Pika 1.0: video generation and editing from text, images and video

Pika launches Pika 1.0, a new model and web platform for generating and editing video in many styles, and announces a $35M Series A led by Lightspeed, for $55M raised in total. The company claims over 500,000 users.

Image & Video Gen Pika 1.0text-to-videoconsumer AI

November 28, 2023 High

Amazon Q in preview: AWS's AI assistant for developers and IT

At re:Invent 2023 AWS previews Amazon Q: chat in the AWS console and IDEs, console error troubleshooting, EC2 instance recommendations, feature development with /dev and Code Transformation to upgrade Java 8 and 11 apps to Java 17.

AI Coding AWSAmazon Qre:Invent

November 21, 2023 High

Claude 2.1: 200K context and fewer false statements

Anthropic ships Claude 2.1: 200K-token context window (about 150,000 words, over 500 pages), a 2x decrease in false statements versus Claude 2.0, tool use in beta, system prompts and the Workbench in the Console.

Foundation Models AnthropicClaude 2.1200K context

November 15, 2023 Medium

Microsoft unveils SharePoint Premium: Syntex becomes the AI content platform

At Ignite 2023 Microsoft announces SharePoint Premium, the new name for Microsoft Syntex: AI document-processing services, new content experiences and governance tools on top of SharePoint.

Enterprise AI MicrosoftSharePointSharePoint Premium

November 14, 2023 High

CrewAI: AI agent teams with roles, goals and backstories like an office

João Moura publishes the first CrewAI release on PyPI (0.1.0, 14 November 2023): a Python framework built on LangChain for orchestrating teams of LLM agents, each with a role, goal and backstory.

Agents CrewAIMulti-AgentRoles

November 14, 2023 Medium

Notion AI Q&A: natural-language questions across the whole workspace

Notion launches Q&A in beta, a Notion AI feature that answers questions using workspace content (docs, projects, meeting notes), drawing only on pages the user has permission to view.

Enterprise AI NotionNotion AIKnowledge Base

November 6, 2023 Medium

MITRE and Microsoft update ATLAS with a generative AI focus

MITRE and Microsoft add a generative AI focus to the ATLAS knowledge base: new tactics and techniques for LLM-based systems and case studies from 2023 incidents such as the ChatGPT plugin leak and PoisonGPT.

AI Security MITREATLASAdversarial ML

November 6, 2023 Medium

Whisper Large v3: multilingual ASR trained on 5 million hours

OpenAI releases Whisper large-v3 (November 6, 2023, the day of DevDay): same architecture as large-v2, 128 mel bins instead of 80, a Cantonese token and a 10-20% error reduction compared to large-v2. Weights on Hugging Face under Apache-2.0.

Voice & Audio Whisper Large v3ASRspeech recognition

November 6, 2023 High

OpenAI DevDay: GPT-4 Turbo, GPTs and the Assistants API

At OpenAI's first developer conference: GPT-4 Turbo (128K context, input 3x and output 2x cheaper than GPT-4), GPTs (custom versions of ChatGPT) and the Assistants API in beta, plus vision, DALL-E 3 and TTS in the API.

Foundation Models OpenAIDevDayGPT-4 Turbo

November 6, 2023 High

OpenAI launches the TTS API: six voices and two models, tts-1 and tts-1-hd

At DevDay OpenAI opens text-to-speech in its API: six preset voices, the tts-1 model optimized for real time and tts-1-hd for quality, starting at $0.015 per 1,000 characters.

Voice & Audio OpenAITTSAPI

November 6, 2023 High

S-LoRA and Punica: thousands of LoRA adapters served from a single base model

Between October and November 2023 Punica (University of Washington and Duke) and S-LoRA (Sheng et al., Stoica and Gonzalez's group at Berkeley) are published: two systems for serving many LoRA adapters on top of a single copy of the base model. In January 2024 multi-LoRA support with the Punica kernels lands in vLLM.

AI Infrastructure LoRAS-LoRAPunica

November 4, 2023 Medium

Grok: xAI's first chatbot, with real-time access to X

Elon Musk's xAI introduces Grok, built on the Grok-1 model: real-time knowledge via the X platform, initial access for a few US users and then for X Premium+ subscribers. In its reported benchmarks it beats GPT-3.5 and Inflection-1 but stays below GPT-4.

Foundation Models GrokGrok-1xAI

November 3, 2023 Medium

DeepSpeed-FastGen: Dynamic SplitFuse for LLM serving with up to 2.3x vLLM's throughput

On November 3, 2023 Microsoft's DeepSpeed team introduces FastGen, an LLM serving system built on DeepSpeed-MII and DeepSpeed-Inference. With Dynamic SplitFuse scheduling it claims up to 2.3x effective throughput versus vLLM and up to 3.7x lower tail latency.

AI Infrastructure DeepSpeedFastGenMII

November 2, 2023 High

DeepSeek-Coder: open code models from 1.3B to 33B parameters

DeepSeek releases DeepSeek-Coder in four sizes (1.3B, 5.7B, 6.7B and 33B), trained from scratch on 2 trillion tokens, 87% of them code, with a 16K context and fill-in-the-middle; the 33B model beats CodeLlama-34B on the main coding benchmarks.

AI Coding DeepSeek-Codercode modelFIM

November 2, 2023 High

Yi-34B and Yi-6B: 01.ai releases its first bilingual EN/ZH open-weight models

On 2 November 2023 01.ai, Kai-Fu Lee's startup, releases the Yi-6B and Yi-34B base models, trained on 3T English and Chinese tokens, with a 4K context and 200K variants following on 5 November.

Foundation Models Yi-34B01.aiKai-Fu Lee

November 2, 2023 Medium

RoboGen: an agent that generates its own tasks, scenes and supervision to train robots in simulation

Researchers from CMU, Tsinghua, MIT, UMass Amherst and MIT-IBM AI Lab introduce RoboGen: it uses foundation and generative models to propose robot tasks, build the simulated environments and generate the supervision to learn skills, with little human involvement.

Robotics RoboGenGenerative SimulationSynthetic Data

November 1, 2023 Landmark

Bletchley AI Safety Summit: 28 countries and the EU sign a declaration on frontier AI risks

November 1-2, 2023: at the first AI Safety Summit, at Bletchley Park, 28 countries plus the European Union sign the Bletchley Declaration, acknowledging the potential for serious, even catastrophic, harm from frontier AI. During the summit the UK and the US announce their respective AI Safety Institutes.

AI Security BletchleyAI Safety Summitinternational

November 1, 2023 Landmark

Microsoft 365 Copilot GA: available at 30 dollars per user per month

On November 1, 2023 Microsoft 365 Copilot (then Copilot for Microsoft 365) becomes available to enterprise customers at 30 dollars per user per month, with Microsoft 365 Chat as its main experience.

Enterprise AI Microsoft 365CopilotGA

October 2023

October 30, 2023 Landmark

Executive Order 14110: the first major US executive order on safe AI

Biden signs Executive Order 14110 on AI: the largest models (10^26-operation threshold) must be reported to the government with red-team test results shared, NIST red-teaming guidelines, standards for labeling synthetic content, and measures to attract AI talent.

AI Security Executive OrderBidenAI safety

October 25, 2023 High

Zephyr-7B: DPO on Mistral 7B beats Llama-2-70B-chat on MT-Bench

Hugging Face (H4 team) trains Zephyr-7B-beta with distilled SFT + Direct Preference Optimization on Mistral 7B: 7.34 on MT-Bench versus 6.86 for Llama-2-70B-chat, with 10x fewer parameters. MIT license.

Foundation Models ZephyrHuggingFaceDPO

October 19, 2023 High

NVIDIA TensorRT-LLM goes public and open source: LLMs optimized for NVIDIA GPUs

On October 19, 2023 NVIDIA makes the open-source TensorRT-LLM library (v0.5.0), announced in September, public on GitHub: a Python API to build optimized TensorRT engines for LLMs, with FP8, INT4/INT8 quantization, in-flight batching and tensor/pipeline parallelism.

AI Infrastructure NVIDIATensorRT-LLMFP8

October 19, 2023 High

Eureka: NVIDIA uses GPT-4 to write robot reward functions

NVIDIA, UPenn, Caltech and UT Austin present Eureka: GPT-4 writes and iteratively improves reward-function code for reinforcement learning, beating expert-written rewards on 83% of 29 tasks and teaching a simulated robot hand to spin a pen.

Robotics EurekaNVIDIAreward function

October 18, 2023 High

Amazon puts Agility Robotics' Digit humanoid to the test

Amazon announces it will test Digit, Agility Robotics' bipedal robot, in its operations: the first task is moving empty totes, at its R&D facility south of Seattle.

Robotics Agility RoboticsDigitHumanoid Robot

October 16, 2023 Medium

OpenAgents: an open platform for using language agents from the browser

XLang Lab (University of Hong Kong) publishes the OpenAgents paper, an open-source platform with three agents (data, plugins, web) usable from a web interface and deployable locally. The code had been released on October 13 under Apache-2.0.

Agents OpenAgentsXLangdata agent

October 12, 2023 Medium

Lakera Guard: an API against prompt injection and data leaks in LLMs

Swiss startup Lakera officially launches Lakera Guard, an API that applications integrate to protect LLMs from prompt injection, jailbreaks, personal data leaks, inappropriate content and hallucinations. It builds on the attack database collected through the Gandalf game.

AI Security LakeraPrompt InjectionJailbreak

October 12, 2023 High

PAIR: automated LLM-vs-LLM jailbreaking in under twenty queries

University of Pennsylvania researchers present PAIR: an attacker LLM iteratively refines its prompts against a target LLM with only black-box access, often finding a jailbreak in fewer than twenty queries and with no human in the loop.

AI Security PAIRjailbreakautomated

October 6, 2023 High

Latent Consistency Models: Stable Diffusion images in 2-4 steps

Tsinghua University researchers present Latent Consistency Models: distilling a latent diffusion model such as Stable Diffusion yields 768x768 images in 2-4 inference steps instead of dozens, with about 32 A100 GPU hours of training.

Image & Video Gen LCMlatent consistencydistillation

October 5, 2023 High

CogVLM: a visual expert inside every layer to keep language skills intact

The THUDM group (Tsinghua / Zhipu AI) releases CogVLM-17B: 10 billion visual plus 7 billion language parameters, with a visual expert module in the attention and FFN layers and SOTA results on 10 classic cross-modal benchmarks.

Multimodal AI CogVLMVisual ExpertVQA

October 5, 2023 High

LLaVA-1.5: open-source vision-language, SOTA on 11 benchmarks with 1.2M public data points

LLaVA-1.5 combines CLIP ViT-L/14 at 336px, an MLP projector and Vicuna v1.5: state-of-the-art results on 11 benchmarks with about 1.2 million public data points and one day of training on a single 8-A100 node.

Multimodal AI LLaVAVision-LanguageCLIP

October 3, 2023 High

Open X-Embodiment and RT-X: a shared robotics dataset from 22 robot types

Google DeepMind and partners from 33 academic labs pool over one million trajectories from 22 robot types (527 skills) and train RT-1-X and RT-2-X, which improve over models trained on a single robot.

Robotics Google DeepMindOpen X-EmbodimentDataset

September 2023

September 27, 2023 High

Mistral 7B: Europe joins the open-source race

Mistral AI (Paris), founded in April 2023 by former Meta and DeepMind researchers, releases Mistral 7B under Apache 2.0. According to the company it beats Llama 2 13B on all evaluated benchmarks with just over half the parameters.

Open Source Models MistralMistral 7BOpen Source

September 27, 2023 Medium

Workday Rising 2023: generative AI comes to HR and Finance

At its Workday Rising conference, Workday announces generative AI features for HR and Finance, including Developer Copilot for Workday Extend, job description generation, contract analysis and automated collection letters.

Enterprise AI WorkdayWorkday RisingHR AI

September 26, 2023 Medium

Copilot in Windows 11: the AI assistant enters the operating system

From September 26, 2023 Microsoft rolls out Copilot in Windows in preview with the free Windows 11 version 22H2 update, announced on September 21 alongside the unified Microsoft Copilot brand.

Enterprise AI MicrosoftCopilotWindows 11

September 26, 2023 High

SAP announces Joule, the generative AI copilot for the SAP cloud portfolio

On 26 September 2023 SAP announces Joule, a natural-language assistant to be embedded in its cloud applications: first SuccessFactors and SAP Start in late 2023, then S/4HANA Cloud public edition in early 2024.

Enterprise AI SAPJouleERP

September 25, 2023 High

Anthropic + Amazon: up to 4 billion dollars, AWS becomes Claude's primary cloud

On September 25, 2023 Amazon announced an investment of up to 4 billion dollars in Anthropic (1.25 billion upfront, minority stake). AWS becomes Anthropic's primary cloud provider; Anthropic will use Trainium and Inferentia chips and expands Claude support on Amazon Bedrock.

Enterprise AI AnthropicAWSClaude

September 25, 2023 High

ChatGPT can see, hear, and speak: voice and images arrive

On September 25, 2023 OpenAI announces voice and images in ChatGPT for Plus and Enterprise users, rolling out over the following two weeks: voice conversations on iOS and Android with five voices, image analysis on all platforms.

Multimodal AI OpenAIChatGPTvoice

September 23, 2023 Landmark

Ollama 0.1: pull and run local LLMs with one command, now on Linux too

On September 23, 2023 Ollama v0.1.0 ships: after the July Mac preview, the tool to download and run LLMs locally with a single command comes to Linux, with NVIDIA GPU acceleration on by default.

Local AI OllamaCLILLM Locale

September 20, 2023 High

DALL-E 3: images that actually follow instructions

On September 20, 2023 OpenAI announced DALL-E 3, built into ChatGPT: it follows prompts much better than DALL-E 2, especially long ones, handles text and hands better, refuses the styles of living artists and lets artists opt their work out of future training.

Image & Video Gen OpenAIDALL-E 3Text-to-Image

September 13, 2023 High

XTTS: Coqui AI's multilingual zero-shot voice cloning

Coqui AI adds XTTS to its TTS library (v0.17.0, September 2023): cross-language voice cloning from a 6-second clip, with downloadable weights under the Coqui Public Model License. The November 2023 v2 reaches 16 languages.

Voice & Audio XTTSCoquimultilingual

September 13, 2023 High

Adobe Firefly becomes commercially available: web app, Photoshop, Illustrator and Express

On September 13, 2023 Adobe takes Firefly out of beta: available for commercial use in the web app, in Creative Cloud (Generative Fill in Photoshop) and in Adobe Express, with a Generative Credits system.

Image & Video Gen Adobe FireflyGenerative FillLicensed Content

September 12, 2023 Medium

ExLlamaV2: quantized LLM inference on consumer GPUs with the EXL2 format

ExLlamaV2, turboderp's inference library for consumer GPUs, introduces the EXL2 format: 2, 3, 4, 5, 6 and 8-bit quantization that can be mixed to reach any average bitrate between 2 and 8 bits per weight. At 2.55 bpw Llama 2 70B runs on a single 24 GB GPU.

AI Infrastructure ExLlamaV2EXL2Quantizzazione

September 11, 2023 High

Phi-1.5: 1.3 billion parameters trained on textbook-style synthetic data

Microsoft Research releases phi-1.5, a 1.3B-parameter model trained mostly on synthetic textbook-style data: it is comparable to 7B models on common sense and beats Llama 2-7B on GSM8K and HumanEval.

Foundation Models Phi-1.5small language modelsynthetic data

September 11, 2023 High

Medusa: extra decoding heads instead of a draft model, roughly 2x faster

Tianle Cai, Yuhong Li, Tri Dao and colleagues present Medusa: a few decoding heads added to the main model predict several future tokens, verified with tree attention. Roughly 2x speedup on Vicuna with no second draft model.

AI Infrastructure MedusaSpeculative DecodingMulti-Head

September 6, 2023 Medium

Slack AI announced: channel and thread summaries plus search answers

Ahead of Dreamforce 2023, Salesforce announces Slack AI, generative AI built into Slack with three features: channel recaps, thread summaries and search answers. A pilot follows, with launch planned for 2024.

Enterprise AI SlackSalesforceProductivity

September 6, 2023 High

Falcon-180B: the largest open-weights model at release

Abu Dhabi's Technology Innovation Institute releases Falcon-180B on Hugging Face, 180 billion parameters trained on 3.5 trillion tokens: top pre-trained open model on the Open LLM Leaderboard, under a license that allows commercial use but excludes hosting use.

Foundation Models Falcon-180BTIIopen weights

September 5, 2023 High

Open Interpreter 0.1.0: an LLM that executes code locally

On September 5, 2023 Open Interpreter 0.1.0 ships, an open-source implementation of ChatGPT's Code Interpreter that lets an LLM write and run Python, JavaScript and Shell code directly on your computer, asking for confirmation before each execution.

Local AI Open InterpreterCode ExecutionLLM

September 5, 2023 Medium

Zoom AI Companion: an AI assistant at no extra cost on paid accounts

Zoom introduces AI Companion (formerly Zoom IQ), included at no additional cost on paid accounts: meeting summaries, in-meeting catch-up and message drafting in Team Chat, with more features planned through spring 2024.

Enterprise AI ZoomAI CompanionMeeting AI

August 2023

August 29, 2023 Medium

Google Cloud TPU v5e: a cost-focused accelerator for training and inference

At Google Cloud Next '23 Google announces Cloud TPU v5e in preview: up to 2x training performance per dollar and up to 2.5x inference performance per dollar versus TPU v4, with pods of up to 256 chips. Alongside it come general availability of Cloud TPUs in GKE and Multislice in preview.

AI Infrastructure TPU v5eGoogle Cloudinference

August 28, 2023 Medium

ChatGPT Enterprise: unlimited GPT-4 and business data kept out of training

OpenAI launches ChatGPT Enterprise: GPT-4 with no usage caps and up to 2x faster, 32K-token context, unlimited advanced data analysis, SOC 2, encryption, an admin console with SSO. Business data is not used to train the models.

Enterprise AI OpenAIChatGPT EnterpriseGPT-4

August 26, 2023 Medium

WizardCoder-Python-34B: 73.2% on HumanEval with Evol-Instruct applied to code

The WizardLM team releases WizardCoder-Python-34B-V1.0, scoring 73.2% pass@1 on HumanEval and, according to the authors, surpassing GPT-4 (March 15, 2023 version), ChatGPT-3.5 and Claude 2. It follows June's WizardCoder-15B, introduced in the paper that adapts Evol-Instruct to code.

AI Coding WizardCoderEvol-InstructHumanEval

August 24, 2023 High

Code Llama: Meta's open-weights model for code

On August 24, 2023 Meta released Code Llama (7B, 13B, 34B), Llama 2 specialized for code with 500 billion additional tokens. Three variants: base, Python and Instruct. Same community license as Llama 2, which allows commercial use.

AI Coding MetaCode LlamaOpen Weights

August 23, 2023 Medium

Apptronik unveils Apollo, a humanoid for warehouses and factories

On August 23, 2023 Texas-based Apptronik unveils Apollo: an electric humanoid of about 1.72 m and 72.5 kg, 25 kg payload, swappable battery lasting about 4 hours. First use planned in logistics, with commercial production targeted by end of 2024.

Robotics ApptronikApolloHumanoid Robot

August 22, 2023 High

ElevenLabs exits beta and launches Eleven Multilingual v2

After opening its beta in January 2023, ElevenLabs ends the beta phase and releases Eleven Multilingual v2, a speech synthesis model for 28 languages ("nearly 30"), and opens Professional Voice Cloning to all creators. It reports over one million users.

Voice & Audio ElevenLabsVoice CloningTTS

August 22, 2023 High

IDEFICS: Hugging Face releases an open-access reproduction of Flamingo

Hugging Face publishes IDEFICS, an open-access reproduction of DeepMind's Flamingo in 9B and 80B sizes (base and instruct), built on LLaMA v1 and OpenCLIP and trained on public data including the new OBELICS dataset.

Multimodal AI VLMOpen SourceFew-Shot Learning

August 22, 2023 High

SeamlessM4T: Meta's single model for speech and text translation in about 100 languages

On August 22, 2023 Meta introduced SeamlessM4T, a single model for speech recognition and text-to-text, speech-to-text, text-to-speech and speech-to-speech translation across nearly 100 languages (35 languages plus English for speech output). Weights released under the non-commercial CC BY-NC 4.0 license.

Voice & Audio SeamlessM4TMetaspeech translation

August 22, 2023 Medium

Qwen-VL and Qwen-VL-Chat: Alibaba's open VLMs with grounding and Chinese-English OCR

Alibaba's Qwen team releases Qwen-VL and Qwen-VL-Chat, vision-language models built on Qwen-7B with 448x448 input, reading of Chinese and English text in images, and object localization through bounding box output.

Multimodal AI VLMOCRDocument Understanding

August 21, 2023 Medium

GGUF: llama.cpp's new file format for local models

llama.cpp adopts GGUF, a new file format for models run with GGML that replaces the earlier formats (GGML, GGMF, GGJT) with a single self-describing file and extensible key-value metadata.

AI Infrastructure GGUFGGMLQuantizzazione

August 16, 2023 High

AutoGen: LLM agents that converse with each other to solve a task

A technical report from Microsoft, Penn State and the University of Washington introduces AutoGen, an open-source framework in which multiple customizable agents, backed by LLMs, tools or human input, converse with each other to complete a task.

Agents AutoGenmulti-agentMicrosoft Research

August 13, 2023 Medium

IP-Adapter: an image as a prompt for Stable Diffusion

Tencent AI Lab releases IP-Adapter, a 22M-parameter adapter that gives text-to-image diffusion models the ability to use an image as a prompt, without modifying the base model. Code and models on GitHub under Apache 2.0.

Image & Video Gen TencentIP-AdapterStable Diffusion

August 8, 2023 Medium

Continue: the open source VS Code extension that lets you pick the model

Continue (YC S23) launches its open source VS Code extension on Product Hunt, billed as the «open-source coding autopilot»: ChatGPT-style assistance inside the IDE, with the freedom to pick the model, open source ones included.

AI Coding ContinueOpen SourceIDE Extension

August 7, 2023 Medium

AgentBench: a benchmark for evaluating LLMs as agents

A Tsinghua-led team releases AgentBench, a benchmark with 8 distinct interactive environments for evaluating LLMs as agents. Across more than 25 models, top commercial LLMs perform well, but a clear gap remains with open-source models.

Agents TsinghuaAgentBenchBenchmark

August 2, 2023 Medium

OpenFlamingo: the paper on the open Flamingo reproduction, models from 3B to 9B

The OpenFlamingo team (University of Washington, LAION and others) publishes the technical report on a family of autoregressive vision-language models from 3B to 9B parameters, an open reproduction of DeepMind's Flamingo. Across seven datasets they average 80-89% of the corresponding Flamingo performance. Code (MIT) and weights are public.

Multimodal AI OpenFlamingoFlamingoopen source

August 2, 2023 High

Meta AudioCraft: open framework for generating music and sound from text

On August 2, 2023 Meta released AudioCraft, bringing together MusicGen (text-to-music), AudioGen (text-to-sound effects) and EnCodec (neural codec with an improved decoder). Code under the MIT license, model weights under the non-commercial CC-BY-NC 4.0 license.

Voice & Audio MetaAudioCraftMusicGen

August 1, 2023 High

MetaGPT: agents with company roles that write software together

The MetaGPT paper (Hong et al., DeepWisdom and others) describes a multi-agent framework that encodes a software company's standard operating procedures: each LLM agent has a role (Product Manager, Architect, Project Manager, Engineer, QA Engineer) and passes structured documents to the next.

Agents MetaGPTMulti-AgentSoftware Engineering

August 1, 2023 High

OWASP LLM Top 10: the 10 critical vulnerabilities in AI applications

OWASP publishes version 1.0 of its Top 10 for LLM applications: ten risk categories, from prompt injection to insecure output handling, chosen with input from almost 500 experts.

AI Security OWASPLLM Top 10Vulnerabilità

July 2023

July 28, 2023 High

RT-2: the robot that reasons with a language model

Google DeepMind's RT-2 is a vision-language-action model: a VLM co-trained on web data and robot data, with actions written as text tokens. On unseen scenarios it goes from RT-1's 32% to 62%.

Robotics DeepMindRT-2VLA

July 27, 2023 High

Universal adversarial attacks on LLMs: jailbreak suffixes that transfer to ChatGPT, Bard and Claude

Zou et al. (CMU, Center for AI Safety, Google DeepMind, Bosch) present GCG: suffixes automatically optimized on open-source models such as Vicuna that, appended to a harmful request, bypass refusals and also transfer to ChatGPT, Bard, Claude and LLaMA-2-Chat.

AI Security JailbreakAdversarial AttackCMU

July 26, 2023 High

Stable Diffusion XL 1.0: the quality jump for open-weights image models

On July 26, 2023 Stability AI released SDXL 1.0: a 3.5-billion-parameter base model (6.6 billion for the full pipeline with refiner), native 1024×1024 resolution, public weights under the CreativeML Open RAIL++-M license.

Image & Video Gen Stability AISDXLStable Diffusion

July 18, 2023 Landmark

Llama 2: weights become commercially usable

Meta releases Llama 2 (7B, 13B, 70B, base and Chat) under a license that allows commercial use; a separate permission is needed only above 700 million monthly active users. A competitive LLM becomes deployable to production without depending on an API.

Open Source Models MetaLlama 2Open Weights

July 17, 2023 High

FlashAttention-2: complete rewrite, about 2x faster, with MQA/GQA and head-dim up to 256

On July 17, 2023 Tri Dao published FlashAttention-2: about 2x faster than FlashAttention, 50-73% of theoretical FLOPs on A100 (vs 25-40%), parallelism along sequence length too, MQA/GQA support and head dimensions up to 256.

AI Infrastructure FlashAttention-2AttentionTransformer

July 14, 2023 Medium

Sweep: the AI «junior developer» that turns GitHub issues into pull requests

Sweep (YC S23) launches on Product Hunt its open source agent that reads the codebase, plans changes and opens a pull request from bug reports and feature requests written in GitHub issues.

AI Coding Code AgentGitHubPR Automation

July 13, 2023 High

WormGPT: an unfiltered chatbot sold to cybercriminals

On July 13, 2023 SlashNext describes WormGPT, a chatbot based on the open-source GPT-J model promoted on a hacker forum as a ChatGPT alternative with no ethical limits; in the researchers' test it writes a highly convincing BEC fraud email.

AI Security WormGPTdark LLMcybercrime

July 11, 2023 High

Claude 2: 100K-token context and claude.ai opens to the public

Anthropic launches Claude 2, available via API and on the new claude.ai site in public beta in the US and UK, with 100,000 tokens of context per prompt and improvements in coding, math and reasoning over Claude 1.3.

Foundation Models AnthropicClaude 2100K Context

July 11, 2023 Medium

IBM makes watsonx.ai and watsonx.data available to enterprises

On July 11, 2023 IBM makes available watsonx.ai, a studio for foundation models, generative AI and machine learning with IBM and Hugging Face community models, and watsonx.data, a data store for enterprise data; watsonx.governance is announced for later in the year.

Enterprise AI IBMwatsonxwatsonx.ai

July 10, 2023 High

AnimateDiff: bringing motion to personalized Stable Diffusion models

Shanghai AI Lab, CUHK and Stanford present AnimateDiff: a motion module trained once on video and plugged into personalized models derived from Stable Diffusion (DreamBooth, LoRA), which can then generate short animations without model-specific tuning.

Image & Video Gen AnimateDiffmotion moduleStable Diffusion

June 2023

June 22, 2023 High

Stable Diffusion XL 0.9: 1024x1024 and a base + refiner pipeline

On June 22, 2023 Stability AI releases SDXL 0.9: a 3.5-billion-parameter base model plus a 6.6-billion-parameter ensemble pipeline, 1024x1024 images, research-only weights under a non-commercial license.

Image & Video Gen Stable Diffusion XLSDXLStability AI

June 22, 2023 High

AudioPaLM: Google fuses PaLM-2 and AudioLM into a model that listens and speaks

Google presents AudioPaLM, which merges the PaLM-2 text model and the AudioLM speech model into a shared vocabulary of text and audio tokens: state-of-the-art speech translation results and speech-to-speech translation that keeps the speaker's voice.

Voice & Audio AudioPaLMGoogleaudio LLM

June 20, 2023 High

vLLM: 24x LLM throughput with PagedAttention from UC Berkeley

A UC Berkeley team releases vLLM, an open source LLM serving library built on PagedAttention, which manages the KV cache like OS virtual memory: up to 24x the throughput of HuggingFace Transformers and up to 3.5x that of TGI.

AI Infrastructure vLLMBerkeleyPagedAttention

June 20, 2023 High

RoboCat: DeepMind's robotic agent that generates its own training data

Google DeepMind introduces RoboCat, a Gato-based agent that adapts to new tasks and new robot arms with 100-1000 demonstrations and generates data for its own next version: on unseen tasks average success rises from 36% to 74%.

Robotics RoboCatDeepMindself-improvement

June 20, 2023 High

Phi-1: 1.3 billion parameters and 50.6% on HumanEval

Microsoft Research publishes 'Textbooks Are All You Need' and phi-1, a 1.3B-parameter Python model trained on textbook-quality data filtered from the web and generated with GPT-3.5: 50.6% pass@1 on HumanEval and 55.5% on MBPP.

Foundation Models Phi-1MicrosoftSmall Models

June 16, 2023 High

Voicebox: Meta brings flow matching to TTS with audio editing and 6 languages

Meta unveils Voicebox, a non-autoregressive speech model based on flow matching and trained to infill speech segments: zero-shot synthesis, editing, noise removal and style transfer across 6 languages. The weights are not released.

Voice & Audio VoiceboxTTSFlow Matching

June 13, 2023 High

Function calling: GPT learns to speak JSON

OpenAI adds 'function calling' to the Chat Completions API: developers describe functions with a JSON schema and GPT-4 or GPT-3.5-turbo (0613 versions) can respond with a JSON object stating which function to call and with which arguments.

AI Infrastructure OpenAIFunction CallingTool Use

June 13, 2023 Medium

Garak: the open source vulnerability scanner for LLMs

On June 13, 2023 Leon Derczynski publishes the first alpha of garak, an open source scanner that attacks an LLM with automated probes for prompt injection, jailbreaks, hallucination, data leakage and toxicity. The project is now hosted by NVIDIA.

AI Security GarakVulnerability ScanningRed Teaming

June 10, 2023 Medium

GPT-Engineer: describe a project, the AI asks for clarification and writes the code

Anton Osika shares the demo of GPT-Engineer, an open-source (MIT) project started on GitHub in late April 2023: you write what you want to build in a prompt file, the AI asks clarifying questions and then generates the whole codebase.

Agents GPT-Engineercode generationproject scaffolding

June 8, 2023 High

Adobe brings Firefly and Express to enterprises, with intellectual property indemnity

At Adobe Summit EMEA in London (June 8, 2023) Adobe announces an enterprise offering combining Firefly and Express: IP indemnity for content from certain Firefly workflows, planned training on brand assets, Firefly for Enterprise expected in the second half of 2023.

Enterprise AI AdobeFireflyAdobe Express

June 5, 2023 High

llama.cpp K-quants: super-block quantization and per-tensor mixes

With PR #1684 by Iwan Kawrakow, llama.cpp introduces 2- to 6-bit k-quants (Q2_K-Q6_K) using 256-weight super-blocks, plus S/M/L mixes that give more bits to the most sensitive tensors. On LLaMA 7B, Q4_K_M reaches 5.96 perplexity versus 5.91 for F16 in a 3.8 GB file instead of 13.

Local AI llama.cppK-quantsGGML

June 5, 2023 High

Orca: learning GPT-4 reasoning through explanation traces

Microsoft Research introduces Orca, a 13-billion-parameter model trained on GPT-4's step-by-step explanations (with ChatGPT as teacher assistant): on BigBench Hard it more than doubles Vicuna-13B and reaches parity with ChatGPT.

Foundation Models OrcaMicrosoftImitation Learning

June 5, 2023 Low

Private LLM: a one-time-purchase app for running language models offline on iPhone and Mac

Private LLM, from developer Numen Technologies, reached the App Store for iPhone in June 2023 and later the Mac: language models run on the device, with no connection and no subscription.

Local AI Private LLMiOSmacOS

June 1, 2023 Medium

Microsoft Presidio 2.2.33: batch anonymization and GPT-3 synthetic data

Release 2.2.33 of Presidio, Microsoft's open source framework for detecting and anonymizing personal data in text, adds batch anonymization, the keep operator, a Streamlit demo and a GPT-3 based synthetic data generation example.

AI Security MicrosoftPresidioPII

June 1, 2023 High

AWQ: activation-aware 4-bit weight quantization for LLMs

Song Han's group at MIT publishes AWQ: low-bit weight-only quantization that finds the most important 1% of weights by looking at activations and protects them with per-channel scaling, without backpropagation or reconstruction.

AI Infrastructure AWQQuantizzazione4-bit

May 2023

May 31, 2023 High

Hugging Face LLM Inference Container: TGI comes to Amazon SageMaker

On May 31, 2023 Hugging Face introduces the LLM Inference Container for Amazon SageMaker, built on Text Generation Inference (TGI): continuous batching, tensor parallelism, flash attention and bitsandbytes quantization for serving open LLMs in production.

AI Infrastructure HuggingFaceTGILLM Serving

May 25, 2023 High

Voyager: the agent that learns Minecraft by accumulating skills as code

NVIDIA, Caltech, UT Austin, Stanford and UW Madison present Voyager, a lifelong-learning Minecraft agent that uses GPT-4 to write JavaScript skills and store them in a reusable library.

Agents NVIDIAVoyagerLifelong Learning

May 25, 2023 High

Falcon 40B: TII opens the weights and tops the Open LLM Leaderboard

Abu Dhabi's Technology Innovation Institute releases Falcon 40B, trained on 1 trillion tokens mostly from RefinedWeb; on May 31 the license becomes royalty-free Apache 2.0 and the model ranks first on Hugging Face's Open LLM Leaderboard.

Foundation Models FalconOpen WeightsTII

May 24, 2023 Medium

Gorilla: a fine-tuned LLaMA for writing API calls

UC Berkeley and Microsoft Research present Gorilla, a LLaMA-7B fine-tuned to generate API calls, and the APIBench benchmark. In the paper it beats GPT-4 on this task and, paired with a documentation retriever, reduces hallucinated APIs.

Agents UC BerkeleyGorillaLLaMA

May 24, 2023 High

LM Studio: desktop app to find, download and run local LLMs

In late May 2023 LM Studio 0.0.1 appears: a desktop app (initially Apple Silicon Macs only) to discover, download from Hugging Face and run offline LLaMA-based models in ggml format, powered by llama.cpp.

Local AI LM StudioGGMLllama.cpp

May 23, 2023 High

Microsoft Build 2023: Windows Copilot and OpenAI's plugin standard

At Build 2023 Microsoft announces Windows Copilot (preview from June), adopts OpenAI's open plugin standard for Bing, Dynamics 365 Copilot and Microsoft 365 Copilot, and introduces Microsoft Fabric and Azure AI Studio.

Enterprise AI MicrosoftBuildCopilot

May 23, 2023 Medium

Multi-Agent Debate: several instances of an LLM arguing reason better

MIT and Google Brain researchers show that having several instances of the same LLM propose and debate answers over a few rounds improves reasoning and factual accuracy: with 3 agents and 2 rounds, GSM8K rises from 77% to 85% and MMLU from 63.9% to 71.1%.

Agents multi-agent debatereasoningfactuality

May 17, 2023 High

Tree of Thoughts: the LLM that reasons by exploring alternative branches

Princeton and Google DeepMind propose Tree of Thoughts: the LLM generates and evaluates multiple reasoning paths as a search tree; with GPT-4, Game of 24 success rises from 4% with Chain-of-Thought to 74%.

Agents PrincetonDeepMindTree of Thoughts

May 16, 2023 High

ServiceNow brings generative AI to the Now Platform: Generative AI Controller and Now Assist for Search

At Knowledge 2023 ServiceNow announces the Generative AI Controller, connecting the Now Platform to Azure OpenAI Service and OpenAI API models, and Now Assist for Search, natural-language answers grounded in the customer's knowledge base. Initial release to a limited set of customers.

Enterprise AI ServiceNowNow AssistGenerative AI Controller

May 16, 2023 High

SoundStorm: Google generates 30 seconds of audio in half a second

SoundStorm generates SoundStream codec tokens in parallel from AudioLM semantic tokens: 30 seconds of audio in about 0.5 seconds on a TPU-v4, with AudioLM-level quality and more consistent voice.

Voice & Audio SoundStormAudio GenerationGoogle

May 16, 2023 Medium

SuperAGI: an open-source framework for building and managing autonomous agents

In mid-May 2023 SuperAGI (TransformerOptimus) appears on GitHub: an MIT-licensed open-source framework to build, manage and run autonomous agents, with a graphical interface, installable toolkits and concurrent agents. The first tagged versions (v0.0.1-v0.0.3) arrive by the end of May.

Agents SuperAGIautonomous agentopen source

May 16, 2023 Medium

Sanctuary AI unveils Phoenix, a general-purpose humanoid driven by the Carbon system

On 16 May 2023 Sanctuary AI unveils Phoenix, its sixth-generation general-purpose humanoid robot and the first powered by Carbon, the company's AI control system combining symbolic reasoning and language models with human supervision.

Robotics Sanctuary AIPhoenixCarbon

May 14, 2023 Medium

Rebuff: four-layer open source prompt injection detector with canary tokens

Rebuff is a self-hardening open source framework for detecting prompt injection: heuristics, a dedicated LLM, a vector database of previously seen attacks, and canary tokens to catch leaks. Introduced on the LangChain blog with a ready-made integration.

AI Security RebuffPrompt InjectionDefense

May 11, 2023 High

LocalAI: OpenAI API drop-in replacement with local models

LocalAI, the Go REST server by Ettore Di Giacinto (mudler) compatible with the OpenAI API, reaches version 1.9.0 on May 11, 2023: besides ggml text models it handles audio transcription with whisper.cpp and embeddings. Apps written for OpenAI can point to a local server.

Local AI LocalAIOpenAI APIPrivacy

May 11, 2023 High

InstructBLIP: visual instruction tuning on BLIP-2 with 26 datasets

Salesforce Research, HKUST and NTU extend BLIP-2 with visual instruction tuning: 26 public datasets converted to instructions, state-of-the-art zero-shot results on 13 datasets held out from training, open-source models.

Multimodal AI InstructBLIPInstruction TuningVisual Reasoning

May 10, 2023 High

Google PaLM 2: the model that makes Bard fly

At Google I/O 2023 Google unveils PaLM 2: four sizes (Gecko, Otter, Bison, Unicorn), training on over 100 languages, more than 25 products and features using it. Bard moves to PaLM 2 and drops its waitlist. Med-PaLM 2 and Sec-PaLM arrive too.

Foundation Models GooglePaLM 2Bard

May 10, 2023 Medium

Google MusicLM: text-to-music generation opens to the public

On 10 May 2023 Google lets anyone who signs up to AI Test Kitchen on web, Android and iOS try MusicLM: from a text description it generates two versions of a track. No tracks with specific artists or vocals.

Voice & Audio GoogleMusicLMMusic Generation

May 9, 2023 Medium

TidyBot: a tidying robot that summarizes user preferences with an LLM

A Princeton-led team, with Stanford, Columbia and Google, presents TidyBot: a mobile manipulator that uses an LLM's few-shot summarization to infer from a few examples where a user wants objects put away. 91.2% accuracy on unseen objects in the text benchmark, 85% of objects put away correctly in real-world tests.

Robotics TidyBotPrincetonStanford

May 5, 2023 Medium

MPT-7B: MosaicML releases an open 7B with an Apache 2.0 base and a 65k-token variant

MosaicML releases MPT-7B, trained from scratch on 1T tokens, with the base model under Apache 2.0 and commercial use allowed. Three variants ship alongside it: StoryWriter-65k+ (65k-token context thanks to ALiBi), Instruct (CC-BY-SA-3.0) and Chat (CC-BY-NC-SA-4.0, non-commercial only).

Foundation Models MPT-7BALiBiApache 2.0

May 4, 2023 High

StarCoder: open code model with transparent training data

BigCode, the project led by Hugging Face and ServiceNow, releases StarCoderBase and StarCoder: 15.5 billion parameters, 8K-token context, trained on 1 trillion tokens of permissively licensed code from The Stack, with an opt-out mechanism for authors.

AI Coding StarCoderBigCodeopen source

May 2, 2023 High

privateGPT: chat with your documents, completely offline

imartinez publishes privateGPT: natural-language questions over TXT, PDF and CSV files with a local LLM (GPT4All-J or llama.cpp), embeddings and vector store on your own disk, no data leaving the machine.

Local AI privateGPTRAGPDF Offline

April 2023

April 26, 2023 High

Palantir AIP: demo of an LLM chatbot for military decisions

Palantir presents its Artificial Intelligence Platform (AIP) in a video: in a simulated Eastern European scenario an operator uses a chatbot to request drone reconnaissance, receive three attack options and coordinate jamming of enemy communications. Palantir says AIP can run on classified networks.

Enterprise AI PalantirAIPdefense AI

April 25, 2023 Medium

HuggingChat: Hugging Face launches an open ChatGPT alternative with an open-source Chat UI

Hugging Face launches HuggingChat, a chat prototype powered by OpenAssistant's 30B LLaMA-based model. The interface code, Chat UI, is open source and becomes a base for anyone who wants their own self-hosted chat frontend.

Local AI HuggingChatChat UIopen source

April 24, 2023 Medium

WizardLM: Evol-Instruct evolves instructions for fine-tuning

Microsoft and Peking University present Evol-Instruct, which uses ChatGPT to rewrite instructions into progressively more complex versions, and WizardLM, a LLaMA 7B trained on that data. On the most complex instructions human raters prefer its answers to ChatGPT's.

Foundation Models WizardLMEvol-InstructFine-tuning

April 20, 2023 High

MiniGPT-4 (KAUST): open-source visual chatbot with a single projection layer

KAUST researchers connect a frozen BLIP-2-style visual encoder to the Vicuna language model, also frozen, training only a projection layer. The result describes images in detail and generates a website from a hand-drawn sketch, after a few hours of training.

Multimodal AI MiniGPT-4KAUSTBLIP-2

April 20, 2023 Medium

Bark: Suno releases an open text-to-audio model with laughter, sighs and music

Suno releases Bark, a transformer-based text-to-audio model: besides multilingual speech it generates laughter, sighs, music and other nonverbal sounds marked with tags in the text. Since May 1, 2023 the license is MIT, so it can also be used commercially.

Voice & Audio BarkSuno AITTS

April 19, 2023 Medium

StableLM: Stability AI enters the open LLM race

Stability AI releases StableLM-Alpha at 3B and 7B parameters: base models under CC BY-SA 4.0, fine-tuned versions under a non-commercial license, 4096-token context. 15B to 65B models announced.

Open Source Models Stability AIStableLMopen source

April 17, 2023 High

LLaVA: Visual Instruction Tuning opens the multimodal open-source era

LLaVA connects the CLIP visual encoder to the Vicuna language model and trains it on 158K image instruction examples generated with (text-only) GPT-4. One of the first quality open-source visual assistants.

Multimodal AI LLaVAVisual Instruction TuningOpen Source

April 13, 2023 Medium

Amazon CodeWhisperer GA: free for individuals, with reference tracking

On April 13, 2023 AWS makes CodeWhisperer generally available with a free Individual tier for all developers. It flags when a suggestion resembles open source training code, showing repository and license, and includes security scanning.

AI Coding AmazonCodeWhispererAWS

April 13, 2023 High

AWS Bedrock: managed multi-model AI on Amazon cloud

AWS announces Amazon Bedrock in limited preview: a managed service giving API access to models from AI21 Labs (Jurassic-2), Anthropic (Claude), Stability AI (Stable Diffusion) and the new Amazon Titan. On the same day CodeWhisperer becomes free for individual developers.

AI Infrastructure AWSBedrockmanaged AI

April 13, 2023 High

GPT4All-J (Nomic AI): Apache 2.0 licensed model and desktop client

Nomic AI releases GPT4All-J, an assistant based on GPT-J 6B under the Apache 2.0 license, together with a native desktop chat client with installers for Windows, macOS and Ubuntu, built to run the model locally with no command line.

Local AI GPT4AllGPT4All-JNomic AI

April 7, 2023 High

Generative Agents: 25 AI agents simulate a society in Smallville

Stanford and Google researchers publish Generative Agents: 25 agents powered by ChatGPT (gpt-3.5-turbo) live in a Sims-like virtual village with memory, reflection and planning. From a single seed idea they organize a Valentine's Day party on their own.

Agents StanfordGenerative AgentsSmallville

April 3, 2023 High

BabyAGI: about a hundred lines of Python that sum up the idea of an autonomous agent

Yohei Nakajima publishes BabyAGI, a minimal Python script that uses the OpenAI API and Pinecone to create, reprioritize and execute tasks in a continuous loop. It grew out of his Twitter thread of March 28, 2023.

Agents BabyAGIAutonomous AgentTask Management

March 2023

March 31, 2023 Medium

CAMEL: two LLM agents cooperating through role-playing

KAUST researchers present CAMEL, a role-playing framework where an 'AI user' agent and an 'AI assistant' agent collaborate on a task without human intervention at each step, guided by inception prompting. Open source code.

Agents KAUSTCAMELMulti-Agent

March 30, 2023 Medium

HuggingGPT: ChatGPT as a controller for Hugging Face models

Researchers from Zhejiang University and Microsoft Research Asia present HuggingGPT: ChatGPT plans the task, picks models on Hugging Face based on their descriptions, runs them and summarizes the results. Code in the JARVIS repository.

Agents Microsoft ResearchHuggingGPTJARVIS

March 30, 2023 High

AutoGPT: the first viral AI agent

Toran Bruce Richards (Significant Gravitas) publishes Auto-GPT on GitHub: given a text goal, the program runs GPT-4 in a loop to break it into sub-tasks and execute them on its own. It becomes GitHub's top trending repository.

Agents AutoGPTAgentsOpen Source

March 30, 2023 High

Vicuna-13B: the open chatbot GPT-4 rates above 90% of ChatGPT

A team from UC Berkeley, CMU, Stanford, UC San Diego and MBZUAI fine-tunes LLaMA-13B on about 70,000 conversations taken from ShareGPT. With GPT-4 as the judge, Vicuna-13B reaches more than 90% of ChatGPT and Bard quality, in an evaluation the authors themselves call non-scientific.

Foundation Models VicunaLLaMAfine-tuning

March 28, 2023 High

GPT4All: a ChatGPT-style assistant that runs offline on a laptop CPU

Nomic AI releases GPT4All: LLaMA 7B fine-tuned with LoRA on about 437,000 GPT-3.5-Turbo responses, shipped as a 4-bit quantized checkpoint that runs locally on CPU, together with data, training code and LoRA weights.

Local AI GPT4AllNomic AILLaMA

March 23, 2023 Medium

ChatGPT Plugins: the model starts using external tools

OpenAI introduces plugins for ChatGPT in limited access: two in-house plugins (web browsing and code interpreter), an open source retrieval plugin, and the first third-party plugins such as Expedia, Instacart, KAYAK, Wolfram and Zapier.

Agents OpenAIChatGPTPlugins

March 23, 2023 Medium

Sourcegraph Cody: an AI assistant built on the code graph

Sourcegraph 5.0 introduces Cody, an experimental AI coding assistant that relies on Sourcegraph's code intelligence graph to answer questions about your own code too. A few days later the client is open-sourced under Apache 2.0.

AI Coding SourcegraphCodyCodebase Context

March 22, 2023 High

GitHub Copilot X: in-IDE chat, GPT-4 pull requests and Copilot for the CLI

On March 22, 2023 GitHub introduces Copilot X: Copilot Chat in VS Code and Visual Studio, GPT-4-generated pull request descriptions, Copilot for Docs, Copilot CLI and Copilot Voice, almost all in preview or waitlist.

AI Coding GitHubCopilotChat

March 21, 2023 Medium

Google Bard opens to the public in the US and UK

On March 21, 2023 Google starts opening access to Bard in the US and UK. The chatbot uses a lightweight and optimized version of LaMDA, shows multiple response drafts, and has a 'Google it' button to check answers on Search.

Foundation Models GoogleBardLaMDA

March 20, 2023 High

Reflexion: agents that learn from mistakes without gradient updates

Shinn et al. (Northeastern, MIT, Princeton) propose Reflexion: after each failure the agent writes a natural-language reflection, keeps it in episodic memory and uses it in later attempts, without changing weights. On HumanEval it reaches 91% pass@1 versus 80% for GPT-4.

Agents MITNortheasternPrinceton

March 17, 2023 Medium

Microsoft Semantic Kernel: open source SDK for orchestrating LLMs

Microsoft publishes Semantic Kernel on GitHub, a lightweight open source SDK (MIT license) for combining conventional code and LLM prompts with templating, chaining and planning. At launch C# is the main language, Python is in preview.

Agents Semantic KernelMicrosoftSDK

March 16, 2023 Landmark

Microsoft 365 Copilot: GPT-4 embedded in Word, Excel, Teams and Outlook

Microsoft announces Microsoft 365 Copilot: LLMs, including GPT-4, combined with business data from Microsoft Graph inside Word, Excel, PowerPoint, Outlook and Teams, plus the new Business Chat. For now in testing with a small group of customers.

Enterprise AI Microsoft 365CopilotGPT-4

March 15, 2023 High

PyTorch 2.0 and torch.compile: Graph Compilation Without Rewriting Code

PyTorch 2.0 ships with torch.compile (beta): a function that compiles the model through TorchDynamo and TorchInductor. Across the 163 open-source models in the official benchmark, training on an A100 runs 43% faster on average, and the release stays 100% backward compatible with existing code.

AI Infrastructure PyTorch 2.0torch.compileTorchDynamo

March 14, 2023 High

Claude arrives: the first serious ChatGPT competitor

Anthropic launches Claude, an AI assistant trained with Constitutional AI. Same day as GPT-4. Two versions: Claude (full) and Claude Instant (faster and cheaper).

Foundation Models AnthropicClaudeConstitutional AI

March 14, 2023 High

Google brings generative AI to Workspace, starting with Gmail and Docs

Google announces generative AI in Workspace: drafts and summaries in Gmail, assisted writing in Docs, then images in Slides, formulas in Sheets, and notes in Meet. Trusted testers first, English in the US only.

Enterprise AI Google WorkspaceGenerative AIProductivity

March 14, 2023 Landmark

GPT-4: the reasoning leap that resets the baseline

OpenAI introduces GPT-4, a model that accepts text and image inputs and answers in text. Clearly above GPT-3.5 on reasoning and code; on the Uniform Bar Exam it scores around the 90th percentile.

Foundation Models OpenAIGPT-4Multimodal

March 10, 2023 Landmark

llama.cpp: LLaMA 7B runs 4-bit on MacBook CPU

Georgi Gerganov publishes llama.cpp, a dependency-free C/C++ implementation that runs 4-bit quantized LLaMA 7B on a Mac's CPU: one of the first practical ways to use an LLM of this size offline on a laptop.

Local AI LLaMAllama.cppC++

March 7, 2023 High

Salesforce Einstein GPT: generative AI enters the CRM

Salesforce announces Einstein GPT, which it calls the world's first generative AI for CRM: sales emails, service replies, marketing content, and Apex code generated from customer data. Launched in closed pilot.

Enterprise AI SalesforceEinstein GPTCRM

March 7, 2023 High

Diffusion Policy: robot imitation learning with diffusion models

Researchers from Columbia, Toyota Research Institute and MIT represent a robot's policy as a diffusion process that generates action sequences. On 12 tasks from 4 manipulation benchmarks the method beats the state of the art with an average improvement of 46.9%.

Robotics Diffusion Policyimitation learningdenoising diffusion

March 6, 2023 Landmark

PaLM-E: a 562-billion-parameter embodied language model

Google presents PaLM-E, which feeds images and robot state estimates directly into PaLM's input. The largest version, 562B, plans tasks for real robots and reaches state of the art on OK-VQA.

Robotics GooglePaLM-EVLM

March 1, 2023 High

ChatGPT API: gpt-3.5-turbo at $0.002 per 1K tokens

OpenAI ships the ChatGPT API (gpt-3.5-turbo) at one tenth the price of its earlier GPT-3.5 models, plus the Whisper speech-to-text API at $0.006 per minute.

Foundation Models OpenAIChatGPTAPI

March 1, 2023 Medium

Tesla Optimus at Investor Day 2023: the humanoid robot walks

At its March 1, 2023 Investor Day Tesla shows, in a pre-recorded video, the Optimus humanoid robot walking untethered and doing simple tasks. Musk suggests humanoid robots could one day outnumber humans.

Robotics TeslaOptimusHumanoid Robot

February 2023

February 24, 2023 High

LLaMA: Meta opens foundation models to research

Meta releases LLaMA in four sizes (7B, 13B, 33B, 65B), available to researchers on request. One week later, the weights leak publicly.

Open Source Models MetaLLaMAOpen Weights

February 23, 2023 High

Indirect Prompt Injection: hidden instructions in the data an LLM retrieves

Greshake, Abdelnabi and colleagues (Saarland University, CISPA, sequire technology) show that an attacker can control an LLM-integrated application without talking to it directly, by hiding instructions in content the model will retrieve. The attacks are demonstrated on Bing Chat and on code-completion engines.

AI Security indirect prompt injectionBing ChatLLM-integrated applications

February 10, 2023 High

ControlNet: structural control for Stable Diffusion without retraining

Lvmin Zhang, Anyi Rao, and Maneesh Agrawala (Stanford) introduce ControlNet, a network adding pose, depth, edge, and segmentation control to Stable Diffusion while keeping the base model weights locked.

Image & Video Gen ControlNetStable DiffusionDiffusion Models

February 9, 2023 High

Toolformer: the LLM that learns to use tools on its own

Meta AI presents Toolformer: an LLM that autonomously learns when and how to call external tools (calculator, Wikipedia search, calendar and more) starting from a handful of demonstrations per API.

Agents Meta AIToolformerTool Use

February 7, 2023 Medium

The new Bing: AI chat inside the search engine

Microsoft previews the new Bing and Edge with a chat built on a next-generation OpenAI model customized for search. On March 14, 2023 Microsoft confirms it was GPT-4.

Foundation Models MicrosoftBing ChatPrometheus

February 6, 2023 Medium

Runway Gen-1: transforming an existing video with a prompt or an image

Runway introduces Gen-1, a video-to-video diffusion model that edits existing footage following a text prompt or reference image while preserving its structure. At launch it is limited to a few invited users, opening to the waitlist in the following weeks.

Image & Video Gen Runway Gen-1video-to-videovideo diffusion

February 2, 2023 High

Speculative Sampling: DeepMind speeds up Chinchilla generation 2-2.5x without changing its output

Chen et al. (DeepMind) publish Speculative Sampling: a small draft model proposes short continuations, the large model scores them in parallel and a modified rejection sampling preserves its distribution. On Chinchilla 70B: 2-2.5x faster with no quality loss.

AI Infrastructure Speculative SamplingSpeculative DecodingInference

January 2023

January 30, 2023 High

BLIP-2: the Q-Former bridge between vision and language

Salesforce Research introduces BLIP-2: a lightweight Q-Former bridges a frozen visual encoder and a frozen LLM; on zero-shot VQAv2 it beats Flamingo80B by 8.7 points with 54x fewer trainable parameters.

Multimodal AI BLIP-2Q-FormerImage Captioning

January 26, 2023 High

NIST AI Risk Management Framework 1.0

NIST publishes AI RMF 1.0 (NIST AI 100-1), a voluntary framework for managing the risks of AI systems in organizations, built around four functions: Govern, Map, Measure, Manage.

AI Security NISTAI RMFrisk management

January 17, 2023 Medium

ComfyUI lands on GitHub: a node-based GUI for building Stable Diffusion pipelines

On January 17, 2023 (UTC) the developer known as comfyanonymous publishes the first commit of ComfyUI on GitHub, a Stable Diffusion GUI where the pipeline is designed by connecting nodes in a graph. GPL-3.0 license.

Image & Video Gen ComfyUIStable DiffusionNode-Based

January 16, 2023 Landmark

Azure OpenAI Service goes GA: GPT-3.5, Codex and DALL-E 2 inside Azure

Microsoft brings Azure OpenAI Service to general availability: OpenAI's GPT-3.5, Codex and DALL-E 2 become available to enterprises as an Azure service, backed by the platform's security, compliance and data privacy commitments. ChatGPT is announced as coming soon.

Enterprise AI Azure OpenAIMicrosoftenterprise

January 5, 2023 Landmark

VALL-E: Microsoft clones a voice from 3 seconds of audio using in-context learning

VALL-E imitates an unseen speaker's voice from 3 seconds of audio, without fine-tuning, treating speech synthesis as language modeling over EnCodec tokens. It beats the previous zero-shot state of the art in naturalness and speaker similarity.

Voice & Audio VALL-ETTSVoice Cloning

2022

December 2022

December 21, 2022 High

oobabooga releases text-generation-webui: a web interface for local LLMs

On December 21, 2022 the developer oobabooga publishes text-generation-webui, a Gradio interface for running language models locally (GPT-J-6B, GPT-NeoX-20B, OPT, GALACTICA), with the stated goal of becoming the AUTOMATIC1111 of text generation.

Local AI oobaboogatext-generation-webuilocal LLM

December 15, 2022 Medium

Constitutional AI: an assistant made harmless with feedback from another AI

Anthropic publishes Constitutional AI: the only human oversight on harmlessness is a short list of written principles. The model critiques and rewrites its own answers, then a preference model trained on AI judgments (RLAIF) drives reinforcement learning.

AI Security AnthropicConstitutional AIRLAIF

December 13, 2022 High

Google RT-1: a Transformer for robot control trained on 130,000 real episodes

Robotics at Google and Everyday Robots present RT-1, a robotics transformer trained on over 130,000 episodes collected with 13 robots over 17 months: 97% success on more than 700 seen instructions and 76% on new instructions.

Robotics GoogleRT-1Robotics Transformer

November 2022

November 30, 2022 Landmark

ChatGPT: AI lands in everyone's browser

OpenAI launches ChatGPT, a free conversational interface in research preview, built on a GPT-3.5-series model trained with RLHF. It crosses one million users in five days.

Foundation Models OpenAIChatGPTGPT-3.5

November 24, 2022 Medium

Stable Diffusion 2.0: new OpenCLIP text encoder and 768x768

Stability AI releases Stable Diffusion 2.0 with an OpenCLIP text encoder developed by LAION, 512x512 and 768x768 models, a 4x upscaler, the depth2img model and a new inpainting model.

Image & Video Gen Stable Diffusion 2.0Stability AIOpenCLIP

November 17, 2022 High

Ignore Previous Prompt: the paper studying prompt injection on GPT-3

Fábio Perez and Ian Ribeiro (AE Studio) publish PromptInject, a framework for composing adversarial prompts, and measure two attacks on GPT-3: goal hijacking and prompt leaking.

AI Security Prompt InjectionLLM SecurityAdversarial Attacks

November 16, 2022 Medium

Notion AI alpha: the writing assistant inside the tool you already work in

Notion announces Notion AI in private alpha, available via waitlist only: a writing assistant inside pages for drafts, brainstorming, editing, translation and summaries.

Enterprise AI NotionNotion AIProductivity

November 15, 2022 Medium

Galactica: Meta launches (and pulls in three days) a science LLM

Meta unveils Galactica, a family of models up to 120B parameters trained on 106 billion tokens of scientific text. The public demo is suspended after three days of criticism over false text with an authoritative tone.

Foundation Models MetaGalacticaScience LLM

October 2022

October 31, 2022 High

GPTQ: 3-4 bit post-training quantization for 175-billion-parameter GPT models

Frantar, Ashkboos, Hoefler and Alistarh (IST Austria, ETH Zurich) publish GPTQ: one-shot 3-4 bit quantization without retraining, compressing a 175B model in about 4 GPU hours and running it on a single GPU.

AI Infrastructure GPTQQuantizzazione3-bit

October 28, 2022 Medium

Codeium enters beta: free AI code completion, a Copilot alternative

On October 28, 2022 the startup Exafunction launches the Codeium beta, a free code generation and completion tool. Extensions for JetBrains, Jupyter, Vim/Neovim, Emacs and Chrome follow in the next months.

AI Coding CodeiumExafunctionCode Completion

October 24, 2022 High

EnCodec: Meta AI introduces a high-fidelity neural audio codec

Meta AI publishes EnCodec, a real-time neural audio codec: 1.5-24 kbps for 24 kHz mono and 3-24 kbps for 48 kHz stereo, with MUSHRA results better than Opus, EVS and Lyra-v2. Code and models are public.

Voice & Audio EnCodecNeural CodecAudio Compression

October 24, 2022 Landmark

LangChain: the framework for LLM applications is born

Harrison Chase publishes the first version of LangChain, an open-source Python library for composing LLM calls with prompt templates and chains. It starts as a side project.

Agents LangChainFrameworkLLM Apps

October 20, 2022 High

Flan-T5 and Flan-PaLM: instruction tuning scales to 1,800 tasks

Google scales instruction tuning to 1,836 tasks and up to 540B parameters: Flan-PaLM 540B beats PaLM 540B by 9.4% on average and reaches 75.2% on five-shot MMLU. The Flan-T5 checkpoints are released publicly.

Foundation Models Flan-T5instruction tuningchain-of-thought

October 14, 2022 Medium

Hugging Face Inference Endpoints: Hub models in production on managed infrastructure

Hugging Face presents its new Inference Endpoints service: pick a model from the Hub and publish it as an API on managed cloud infrastructure, with optional autoscaling and three security levels (public, protected, private).

AI Infrastructure Hugging FaceInference EndpointsDeployment

October 6, 2022 Landmark

ReAct: the framework that unites reasoning and acting in LLMs

Yao et al. introduce ReAct, a schema alternating explicit thoughts (Thought) and concrete actions (Act) in LLMs, a schema that will become a reference for LLM-based agents.

Agents ReActReasoningTool Use

October 5, 2022 Medium

Imagen Video and Phenaki: Google answers on text-to-video

Six days after Meta's Make-A-Video, Google publishes two text-to-video papers: Imagen Video, a cascade of diffusion models up to 1280×768 at 24 fps, and Phenaki, which generates variable-length video from a sequence of prompts.

Image & Video Gen GoogleImagen VideoPhenaki

September 2022

September 29, 2022 Medium

Make-A-Video: Meta generates short videos from text starting from an image model

Meta AI presents Make-A-Video, which extends a text-to-image model with temporal modules and learns motion from uncaptioned video, with no need for text-video pairs.

Image & Video Gen MetaMake-A-VideoText-to-Video

September 28, 2022 High

NVIDIA Transformer Engine: the open-source library for FP8 training on Hopper

In September 2022 NVIDIA publishes Transformer Engine on GitHub (first tag v0.1 on September 28), an Apache 2.0 library that brings FP8 on Hopper GPUs (H100) to PyTorch: ready-made modules such as te.Linear and TransformerLayer, an fp8_autocast context and automatic handling of scaling factors.

AI Infrastructure FP8Transformer EngineNVIDIA

September 25, 2022 High

whisper.cpp: offline voice transcription on CPU with pure C/C++

Georgi Gerganov publishes whisper.cpp, a dependency-free C/C++ implementation of OpenAI's Whisper model: it runs on CPU (Mac, Linux, even Raspberry Pi) with mixed F16/F32 precision and low memory, no GPU and no cloud.

Local AI WhisperSpeech-to-TextC++

September 21, 2022 High

Whisper open source: audio transcription becomes a commodity

OpenAI releases Whisper with code and weights under the MIT license: a speech-to-text model trained on 680,000 hours of multilingual audio collected from the web, which also runs locally.

Voice & Audio OpenAIWhisperASR

September 16, 2022 Medium

Character.AI: persona chatbots from the ex-Google minds behind Transformer and LaMDA

Character.AI, founded by Noam Shazeer and Daniel De Freitas, opens its first public beta: a platform where anyone can create AI characters and chat with them.

Foundation Models Character.AIChatbotPersona

September 16, 2022 High

Code as Policies: language models that write robot control code

Robotics at Google shows that LLMs trained on code completion can write Python robot-control programs from natural-language instructions, with no task-specific training, thanks to hierarchical code generation.

Robotics GoogleCode as PoliciesLLM

September 7, 2022 Medium

Replit Ghostwriter: an AI assistant inside the browser IDE

Replit announces Ghostwriter, an AI assistant built into its web IDE: code completion as you type, plus tools to generate, transform and explain code.

AI Coding Code CompletionBrowser IDEAI Assistant

September 7, 2022 Medium

GitHub Copilot: in a controlled experiment the task gets done 55% faster

GitHub publishes research on Copilot: in an experiment with 95 professional developers, those using Copilot wrote an HTTP server in JavaScript 55% faster; a survey of more than 2,000 users also measures satisfaction and mental effort.

AI Coding GitHub CopilotDeveloper ProductivityResearch

September 7, 2022 High

AudioLM: Google teaches a language model to listen and continue audio

AudioLM generates coherent speech and piano continuations using semantic and acoustic tokens, trained on audio only, with no transcripts or scores.

Voice & Audio AudioLMLanguage ModelAudio Generation

August 2022

August 25, 2022 High

DreamBooth: generate your subject in any style with 3-5 photos

Google Research publishes DreamBooth: fine-tune a diffusion model on 3-5 images of a specific subject to reproduce it in any context or style. It becomes one of the core techniques for personalized image generation.

Image & Video Gen DreamBoothpersonalizationfine-tuning

August 22, 2022 Landmark

Stable Diffusion: image generation goes open

Stability AI publicly releases weights and code of a text-to-image latent diffusion model that runs on a consumer GPU. AI image generation leaves the cloud.

Image & Video Gen Stable DiffusionStability AIDiffusion Models

August 16, 2022 High

SayCan: grounding LLMs in robot affordances

Google and Everyday Robots present PaLM-SayCan: combining an LLM for high-level planning with robot value functions that filter only physically executable actions.

Robotics GoogleSayCanEmbodied AI

August 2, 2022 Medium

Textual Inversion: inject a custom concept into diffusion models

On August 2, 2022 researchers from Tel Aviv University and NVIDIA publish Textual Inversion: from 3-5 images it learns a new word in the text embedding space of a frozen text-to-image model, which represents the concept and is used in prompts without modifying the model weights.

Image & Video Gen Textual Inversionpersonalizationembedding

July 2022

July 21, 2022 High

diffusers: Hugging Face launches its library for diffusion models

On July 21, 2022 Hugging Face publishes the first release of diffusers (0.1.2), a modular Python library with pipelines, schedulers and models for diffusion. In August it adds Stable Diffusion support.

Open Source Models Hugging FaceDiffusersLibrary

July 20, 2022 Medium

DALL-E 2 enters beta: invites for one million users and paid credits

OpenAI moves DALL-E 2 into beta and announces it will invite 1 million people from the waitlist over the following weeks, with a free and paid credit system and commercial usage rights for images.

Image & Video Gen OpenAIDALL-E 2Beta

July 12, 2022 High

BLOOM 176B: the first truly open large multilingual LLM

The BigScience collective releases BLOOM, a 176-billion-parameter model trained on 46 human languages and 13 programming languages, under an open RAIL license.

Open Source Models BigScienceBLOOMHugging Face

July 12, 2022 High

Midjourney opens public beta on Discord

On July 12, 2022 Midjourney opens the public beta of its text-to-image generator, used through a Discord bot with the /imagine command.

Image & Video Gen MidjourneyDiscordText-to-Image

June 2022

June 21, 2022 Landmark

GitHub Copilot: AI for code becomes a product for everyone

GitHub announces general availability of Copilot for all developers at $10/month. It's the first mass-market AI tool living inside the daily code editor.

AI Coding GitHubCopilotOpenAI

June 17, 2022 Medium

Unified-IO (AI2): one sequence-to-sequence model for vision, language and multimodal tasks

On June 17, 2022 AI2 and the University of Washington present Unified-IO: a single encoder-decoder Transformer that converts the inputs and outputs of vision, language and multimodal tasks into sequences of discrete tokens, trained jointly on over 90 datasets. It is the first model to perform all 7 tasks of the GRIT benchmark.

Multimodal AI UnifiedIOmultimodalunified model

June 15, 2022 Medium

Tabnine: new generation with per-language models, local, cloud or self-hosted

Tabnine raises $15.5M and presents the new generation of its platform: small per-language models, whole-line and full-function completions, running on the developer's machine, in Tabnine's cloud or self-hosted.

AI Coding TabnineCode CompletionLocal AI

May 2022

May 27, 2022 Landmark

FlashAttention: IO-aware attention that speeds up transformer training

Tri Dao and colleagues (Stanford, University at Buffalo) publish FlashAttention: exact IO-aware attention that avoids writing the attention matrix to HBM, with memory linear in sequence length and up to 3× faster GPT-2 training.

AI Infrastructure FlashAttentionAttentionTransformer

May 23, 2022 High

Imagen: Google enters text-to-image generation

Google Research unveils Imagen, a text-to-image diffusion model that uses a frozen T5 text encoder and reaches a COCO FID of 7.27; in human evaluations on DrawBench it is preferred over DALL-E 2.

Image & Video Gen GoogleImagenText-to-Image

May 12, 2022 High

Gato: DeepMind tries a single agent for 600+ tasks

DeepMind unveils Gato, a 1.2-billion-parameter Transformer that with the same weights plays Atari, captions images, chats and stacks blocks with a real robot arm.

Multimodal AI DeepMindGatoGeneralist Agent

May 10, 2022 Medium

UL2: Google unifies pretraining paradigms with Mixture-of-Denoisers

On May 10, 2022 Google Research published UL2, which merges several pretraining objectives into a single Mixture-of-Denoisers. The 20B UL2 model beats 175B GPT-3 on zero-shot SuperGLUE, and Google releases the checkpoints.

Foundation Models UL2mixture of denoiserspretraining

May 9, 2022 High

NaturalSpeech: Microsoft reaches human-level quality on LJSpeech

Microsoft Research introduces NaturalSpeech, a VAE-based end-to-end TTS system: on LJSpeech it scores -0.01 CMOS against human recordings, a difference that is not statistically significant under the Wilcoxon test (p = 0.69).

Voice & Audio NaturalSpeechMicrosofthuman parity

May 3, 2022 High

Meta OPT-175B: a 175-billion-parameter LLM opened to researchers

Meta AI unveils OPT, a family of models from 125M to 175B parameters: weights up to 30B downloadable, OPT-175B on request for researchers, all under a non-commercial license, with a public training logbook.

Open Source Models MetaOPTOpen Source

April 2022

April 28, 2022 High

DeepMind Flamingo: a visual language model that learns from a few examples

DeepMind presents Flamingo, an 80-billion-parameter model combining images and text: with a few examples in the prompt it beats previous few-shot methods on all 16 tasks evaluated and fine-tuned models on 6 of them.

Multimodal AI Visual Language ModelFew-Shot LearningVQA

April 28, 2022 Medium

Tortoise TTS v2: open-source multi-voice speech synthesis, slow but realistic

James Betker releases Tortoise TTS 2.0, an open-source (Apache 2.0) text-to-speech program that imitates a voice from a few reference clips and targets realistic prosody and intonation, at the cost of very slow generation.

Voice & Audio TTSVoice CloningOpen Source

April 6, 2022 High

DALL·E 2: the quality leap in image generation

OpenAI announces DALL·E 2, a diffusion-based text-to-image generator producing 1024×1024 images, four times its predecessor's resolution, and able to edit and vary existing images. At first it is open only to a small group of users.

Image & Video Gen OpenAIDALL-E 2Diffusion

April 4, 2022 Medium

PaLM 540B: Google's large model trained with Pathways

Google presents PaLM, a 540-billion-parameter dense model trained with the Pathways system on 6144 TPU v4 chips. With chain-of-thought prompting it reaches 58% on GSM8K, above the previous 55% record.

Foundation Models GooglePaLMPathways

March 2022

March 30, 2022 Low

Copilot Labs: GitHub opens an extension for Copilot's experimental features

On March 30, 2022 GitHub Next launched Copilot Labs, a separate VS Code extension for experimental applications of Copilot. At launch it does two things: it explains code in natural language and translates it from one language to another.

AI Coding GitHubCopilot LabsCode Explain

March 29, 2022 Landmark

Chinchilla: the big models were undertrained

DeepMind publishes the Chinchilla paper and shows that, at equal compute, a smaller model trained on many more tokens wins: Chinchilla (70B) beats Gopher (280B), GPT-3, Jurassic-1 and Megatron-Turing NLG.

Foundation Models DeepMindChinchillaScaling Laws

March 22, 2022 Landmark

NVIDIA H100 and Hopper architecture: the foundation-model GPU

At GTC on March 22, 2022 NVIDIA unveils the Hopper architecture and the H100 GPU: 80 billion transistors, HBM3, Transformer Engine with FP8, 4th-gen NVLink. Availability expected from Q3 2022.

AI Infrastructure NVIDIAH100Hopper

March 21, 2022 High

Self-Consistency: sample multiple reasoning paths for better answers

Wang et al. (Google Research, Brain Team) show that sampling several chain-of-thought reasoning paths and picking the most frequent answer improves accuracy on arithmetic and commonsense tasks, e.g. +10% on GSM8K and +24% on MultiArith.

Foundation Models Chain of ThoughtSelf-ConsistencyReasoning

February 2022

February 7, 2022 High

Red Teaming LLMs with LLMs: the DeepMind paper on automated red teaming

Perez et al. (DeepMind) use a language model to automatically generate test cases against a 280B-parameter chatbot, uncovering tens of thousands of offensive replies.

AI Security Red TeamingDeepMindLLM Safety

February 2, 2022 High

AlphaCode: DeepMind takes on competitive programmers

DeepMind unveils AlphaCode, which writes full programs for competitive programming problems and, across 10 Codeforces contests, ranks on average within the top 54.3% of participants, roughly at the median.

AI Coding DeepMindAlphaCodeCompetitive Programming

January 2022

January 27, 2022 High

InstructGPT: human-feedback fine-tuning that teaches GPT-3 to follow instructions

OpenAI presents InstructGPT, GPT-3 refined with RLHF and made the API's default model: labelers prefer the 1.3B-parameter version over the 175B GPT-3 despite it being over 100x smaller.

Foundation Models OpenAIInstructGPTRLHF

2021

December 2021

December 20, 2021 High

GLIDE: OpenAI turns to diffusion for text-to-image

OpenAI publishes GLIDE, a 3.5B-parameter text-to-image diffusion model: classifier-free guidance beats CLIP guidance in human evaluations and its images are preferred over DALL·E's.

Image & Video Gen OpenAIGLIDEDiffusion

December 16, 2021 High

WebGPT: OpenAI teaches GPT-3 to browse the web

OpenAI presents WebGPT, a GPT-3 trained to use a text browser to answer open questions with cited sources; its best version is preferred over the top-voted Reddit answers 69% of the time.

Agents OpenAIWebGPTBrowsing

December 8, 2021 High

Gopher 280B: DeepMind enters the large language model race

DeepMind publishes Gopher, a 280B-parameter dense model, with a 152-task study: it beats the state of the art on 100 of 124 comparable tasks, but scale helps little on logic and math.

Foundation Models DeepMindGopherScaling

December 8, 2021 High

RETRO: DeepMind scales retrieval to 2 trillion tokens

DeepMind publishes RETRO, a Transformer that retrieves passages from a database of up to 2 trillion tokens while generating: the 7.5B-parameter version is comparable to GPT-3 and Jurassic-1 on the Pile with 25x fewer parameters.

Foundation Models DeepMindRETRORetrieval

November 2021

November 18, 2021 High

OpenAI drops the waitlist: GPT-3 API available to all

On November 18, 2021 OpenAI removes the GPT-3 API waitlist: developers in supported countries can sign up and use it right away, while production apps still go through review.

Enterprise AI OpenAIAPIGPT-3

October 2021

October 28, 2021 Medium

Pathways: Google's vision for multitask, multimodal, sparse models

Jeff Dean presents Pathways, the architecture through which Google aims to train a single model able to handle thousands or millions of tasks, multimodal and only partially activated for each request.

AI Infrastructure GooglePathwaysMultitask

October 27, 2021 Medium

Copilot lands on JetBrains and Neovim

At GitHub Universe 2021 GitHub extends the Copilot technical preview to Neovim and JetBrains IDEs, starting with IntelliJ IDEA and PyCharm, and adds multiline completions for Java.

AI Coding GitHubCopilotJetBrains

October 21, 2021 Medium

PyTorch 1.10: CUDA Graphs, stable FX and JIT fusion on CPU

PyTorch 1.10 ships: CUDA Graphs API integration (beta), FX, torch.special and module parametrization become stable, the JIT compiler's automatic fusion reaches CPUs and Android NNAPI support moves to beta.

AI Infrastructure PyTorchFrameworkCUDA Graphs

October 18, 2021 Medium

DeepMind acquires MuJoCo and makes it free

DeepMind announces it has acquired MuJoCo, a physics simulator widely used in robotics and reinforcement learning research, and makes it free to download for everyone, committing to release its code as open source.

Robotics DeepMindMuJoCoPhysics Simulator

October 11, 2021 High

Megatron-Turing NLG 530B: Microsoft and NVIDIA scale dense past GPT-3

Microsoft and NVIDIA announce MT-NLG, a 530-billion-parameter monolithic Transformer trained with DeepSpeed and Megatron-LM on the Selene supercomputer: at the time the largest of its kind, about three times GPT-3.

Foundation Models MicrosoftNVIDIAMegatron

September 2021

September 3, 2021 High

FLAN: instruction tuning that teaches models to follow directions

Google shows that training a model on 60+ tasks framed as instructions dramatically improves zero-shot performance on unseen tasks.

Foundation Models FLANinstruction tuningzero-shot

August 2021

August 16, 2021 High

On the Opportunities and Risks of Foundation Models: Stanford coins the term

Stanford's Center for Research on Foundation Models (CRFM) publishes a sweeping report signed by more than a hundred authors that calls models trained on broad data at scale and adaptable to many tasks foundation models; the term enters common usage.

Foundation Models StanfordCRFMFoundation Models

August 10, 2021 High

Codex API: OpenAI opens access to the model behind Copilot

OpenAI introduces an improved version of Codex, its system that translates natural language into code, and makes it available through the API in private beta, free during the initial period.

AI Coding OpenAICodexAPI

July 2021

July 28, 2021 Medium

OpenAI Triton: writing GPU kernels in a Python-like language

OpenAI releases Triton 1.0, an open-source Python-like language and compiler for writing efficient GPU kernels without CUDA experience: an FP16 matrix multiplication on par with cuBLAS in under 25 lines.

AI Infrastructure OpenAITritonGPU

July 15, 2021 High

AlphaFold 2: open code and database, biology accelerates

On July 15, 2021 DeepMind publishes the AlphaFold 2 method in Nature together with open-source code; on July 22, with EMBL-EBI, it launches a free database of about 350,000 predicted structures, including the ~20,000 human proteins and 20 model organisms.

AI Infrastructure DeepMindAlphaFoldProtein Folding

July 7, 2021 High

Codex paper: OpenAI publishes HumanEval and the model behind Copilot

OpenAI publishes the paper Evaluating Large Language Models Trained on Code, describing Codex (a production version powers GitHub Copilot) and introducing HumanEval, 164 hand-written programming problems with the pass@k metric.

AI Coding OpenAICodexHumanEval

July 7, 2021 High

SoundStream: Google's end-to-end neural audio codec

Google Research introduces SoundStream, a neural audio codec with residual vector quantization (RVQ): a single model runs from 3 to 18 kbps, and at 3 kbps listeners rate it above Opus at 12 kbps.

Voice & Audio SoundStreamneural codecRVQ

June 2021

June 29, 2021 High

GitHub Copilot: autocomplete grows up

GitHub, with OpenAI, launches a technical preview for Visual Studio Code of an assistant that suggests entire lines and functions right in the editor, based on a GPT-3-derived model trained on public code.

AI Coding GitHubCopilotCodex

June 14, 2021 Medium

HuBERT: Facebook AI brings self-supervised learning to speech

Facebook AI publishes HuBERT, a self-supervised speech model that learns by predicting discrete clusters for masked parts of the audio; the roughly 1-billion-parameter version matches or beats wav2vec 2.0 on LibriSpeech and Libri-Light.

Voice & Audio FacebookMetaHuBERT

June 11, 2021 High

VITS: end-to-end TTS with variational autoencoder

VITS merges acoustic model and vocoder into a single end-to-end model: on LJSpeech it reaches a MOS of 4.43, close to real recordings (4.46) and above Tacotron 2 + HiFi-GAN (3.77), with synthesis about 2.4x faster than Glow-TTS + HiFi-GAN.

Voice & Audio VITSTTSend-to-end

June 4, 2021 High

GPT-J 6B: an open model close to GPT-3 Curie

Ben Wang and Aran Komatsuzaki, with EleutherAI, release GPT-J-6B: 6 billion parameters, trained in JAX on TPUs with The Pile, weights under Apache 2.0 and zero-shot results close to GPT-3 Curie (6.7B).

Open Source Models EleutherAIGPT-JOpen Source

June 1, 2021 Medium

Wu Dao 2.0: China announces a 1.75T-parameter model

At the annual conference of BAAI (Beijing Academy of Artificial Intelligence), Wu Dao 2.0 is unveiled: a 1.75-trillion-parameter multimodal Mixture of Experts model, ten times GPT-3 and larger than Switch Transformer's 1.6 trillion.

Foundation Models BAAIWu DaoChina

May 2021

May 28, 2021 Landmark

Anthropic launches: $124M for an AI safety lab

Anthropic, an AI safety and research company led by Dario Amodei (CEO) and Daniela Amodei (President), announces a $124 million Series A led by Jaan Tallinn, to build more reliable, interpretable and steerable AI systems.

AI Security AnthropicAI SafetyFounding

May 18, 2021 Medium

MUM: Google unveils the multitask model for Search

At Google I/O, Google announces MUM (Multitask Unified Model), built on the T5 framework, trained across 75 languages, multimodal (text and images), and, per Google, 1,000 times more powerful than BERT.

Multimodal AI GoogleMUMSearch

May 18, 2021 High

LaMDA: Google unveils its dialogue model

At Google I/O 2021 Sundar Pichai introduces LaMDA (Language Model for Dialogue Applications), a Transformer model trained on dialogue for open-ended conversation; it remains an internal research project.

Foundation Models GoogleLaMDADialogue

April 2021

April 16, 2021 High

MT-Opt: Google trains 7 real robots on 12 tasks with a single multi-task network

Robotics at Google introduces MT-Opt, a multi-task reinforcement learning system: 7 robots collect over 800,000 episodes in 9,600 robot-hours over 57 days, and a single Q-function learns 12 real-world tasks with an average improvement of about 3x over baselines.

Robotics MT-Optmulti-task robot learningoffline RL

April 16, 2021 Medium

HuggingFace Accelerate: One Python Script for CPU, GPU, TPU, and Mixed Precision

On April 16, 2021 Hugging Face introduced Accelerate: by adding a few lines to a standard PyTorch script, the same training code runs on CPU, one or more GPUs (including multi-node) and TPU, with FP16 mixed precision. It will become a building block of many open fine-tuning pipelines.

AI Infrastructure AccelerateHuggingFacemulti-GPU

April 9, 2021 High

Megatron-LM: 3D Parallelism for Training Trillion-Parameter Models

NVIDIA, Stanford and Microsoft Research publish the second Megatron-LM paper: by combining tensor, pipeline and data parallelism they train a 1-trillion-parameter model at 502 petaFLOP/s on 3,072 A100 GPUs, 52% of theoretical per-GPU peak.

AI Infrastructure Megatron-LM3D parallelismpipeline parallelism

March 2021

March 21, 2021 High

GPT-Neo: EleutherAI releases weights for GPT-3-style models

On March 21, 2021 EleutherAI releases pretrained weights for GPT-Neo 1.3B and 2.7B, GPT-3-style models trained on The Pile with Mesh TensorFlow-based code.

Open Source Models EleutherAIGPT-NeoOpen Source

March 9, 2021 Medium

Coqui TTS: first release of the open-source library born from Mozilla TTS

Coqui, a startup founded by former members of Mozilla's speech team, publishes Coqui TTS v0.0.9, the first release of the open-source Python text-to-speech library, with pretrained models for English, Spanish and French.

Voice & Audio CoquiTTSOpen Source

March 7, 2021 High

DeepSpeed ZeRO-3 Offload: over 40 billion parameters on a single GPU

Microsoft ships ZeRO-3 Offload in DeepSpeed, combining ZeRO Stage 3 (partitioning parameters too) with CPU offload: fine-tuning of 40B+ parameter models on a single 32 GB V100 and 2-trillion-parameter models on 512 GPUs.

AI Infrastructure MicrosoftDeepSpeedZeRO-3

January 2021

January 11, 2021 High

Switch Transformer: Google scales to 1.6T parameters with Mixture of Experts

Google publishes Switch Transformer (Fedus, Zoph, Shazeer), a Mixture of Experts model that sends each token to a single expert: up to 7x faster pre-training than T5-Base at equal compute and a variant of about 1.6 trillion parameters.

Foundation Models GoogleMoESparse

January 5, 2021 High

DALL·E and CLIP: OpenAI connects text and images

On January 5, 2021 OpenAI introduces DALL·E, a 12-billion-parameter transformer that generates images from text descriptions, together with CLIP, which learns to connect images and text from 400 million web-scraped pairs.

Multimodal AI OpenAIDALL-ECLIP

2020

December 2020

December 31, 2020 High

The Pile: the open-source 825 GiB dataset for training LLMs

EleutherAI publishes The Pile, an 825 GiB English text corpus made of 22 sub-datasets (filtered Common Crawl, PubMed, Books3, arXiv, GitHub, Stack Exchange and more), designed for pre-training open language models.

Open Source Models EleutherAIThe PileDataset

December 23, 2020 High

MuZero in Nature: mastering games without knowing the rules

DeepMind publishes MuZero in Nature: the agent learns its own model of what matters for planning and, without being given the rules, matches AlphaZero at Go, chess, and shogi and sets a new state of the art on 57 Atari games.

Foundation Models DeepMindMuZeroReinforcement Learning

November 2020

November 30, 2020 Landmark

AlphaFold 2 dominates CASP14: a protein-folding breakthrough

DeepMind announces AlphaFold 2's CASP14 results: median GDT 92.4 across all targets, above the roughly 90 threshold the organizers consider competitive with experimental methods. For the CASP organizers, the single-protein folding problem is largely solved.

Foundation Models DeepMindAlphaFoldCASP

November 12, 2020 Low

fairseq 0.10.0: the first release in nearly a year bundles mBART, wav2vec 2.0 and distributed training

On November 12, 2020 Facebook AI Research released fairseq 0.10.0, the first release since 0.9.0 in December 2019: it brings the code for mBART, wav2vec 2.0 and Linformer, adds model parallel, TPU and ZeRO support, and announces that 1.0 will migrate to Hydra configuration.

Open Source Models MetaFacebook AIfairseq

October 2020

October 22, 2020 Landmark

Vision Transformer (ViT): "An Image is Worth 16x16 Words"

Google Research introduces the Vision Transformer, applying a pure transformer to image patches as if they were tokens, and shows that with pre-training on very large datasets it matches or beats the best CNNs on ImageNet and other benchmarks, with less training compute.

Multimodal AI GoogleVision TransformerViT

October 22, 2020 Medium

mT5: a multilingual T5 over 101 languages

Google Research publishes mT5, a T5 variant pre-trained on mC4 (multilingual Common Crawl) over 101 languages, reaching state of the art on the main cross-lingual benchmarks.

Foundation Models GoogleT5mT5

October 16, 2020 Medium

OpenAI Content Filter: the GPT-3 API moderation classifier

During the GPT-3 API beta OpenAI provides a content filter, documented at least since October 2020, that classifies text as safe, sensitive or unsafe. It is not a dedicated endpoint but an engine (content-filter-alpha) queried through Completions; in 2022 the Moderation API replaces it.

AI Security OpenAIContent FilterSafety

October 13, 2020 Medium

PyTorch Lightning 1.0: a boilerplate-free training loop

On October 13, 2020 PyTorch Lightning 1.0 shipped, the first major version of William Falcon's framework that separates research code (model) from engineering (training loop, distributed, checkpointing, logging). The release unifies logging under self.log, integrates the metrics API and introduces accelerators and plugins.

AI Infrastructure PyTorch LightningOpen SourceTraining Loop

September 2020

September 22, 2020 High

Microsoft gets the exclusive GPT-3 license

Microsoft announces an exclusive license to GPT-3 to use its innovations in its own products and services, while OpenAI keeps offering the model through its own Azure-hosted API.

Enterprise AI MicrosoftOpenAIGPT-3

July 2020

July 29, 2020 Medium

MLPerf Training 0.7: Google records on TPU v3 and a first look at TPU v4

In the MLPerf Training v0.7 results, Google sets records in 6 of 8 benchmarks with a 4,096-chip TPU v3 supercomputer and shows TPU v4 for the first time, on average 2.7x faster than TPU v3 at the same chip count.

AI Infrastructure GoogleTPU v4Pod

July 28, 2020 Medium

Big Bird: sparse attention for 8x longer sequences

Google Research publishes Big Bird, a transformer with sparse attention (local + global + random) that cuts cost from quadratic to linear, handles sequences up to 8x longer on the same hardware, is Turing-complete, and improves QA and summarization on long documents.

Foundation Models GoogleBig BirdSparse Attention

July 7, 2020 High

EleutherAI is founded: a community to replicate GPT-3 in the open

Connor Leahy, Sid Black, and Leo Gao found EleutherAI on Discord with the goal of replicating GPT-3 and releasing models, code, and datasets in the open, kicking off projects like GPT-Neo, GPT-J, and The Pile.

Open Source Models EleutherAIGPT-NeoOpen Source

June 2020

June 30, 2020 High

NVIDIA Triton Inference Server 2.0: the move to version 2 and the KFServing protocols

NVIDIA releases Triton Inference Server 2.0.0 (NGC container 20.06): new HTTP/REST and gRPC protocols based on the KFServing v2 protocol, with a migration from V1 that requires significant changes.

AI Infrastructure NVIDIATritonInference Server

June 29, 2020 Medium

HuggingFace Transformers 3.0: one API for slow and fast tokenizers

HuggingFace releases Transformers 3.0: a new unified API for Python and Rust tokenizers, decoupled padding and truncation, new AutoModel classes per language-model type, MobileBERT, and improved TensorFlow support.

Open Source Models HuggingFaceTransformersTokenizers

June 20, 2020 High

wav2vec 2.0: Facebook AI's "BERT for speech"

Facebook AI publishes wav2vec 2.0, a self-supervised method that learns representations from raw audio: 1.8/3.3 WER on LibriSpeech with all labeled data and 4.8/8.2 with just 10 minutes of transcriptions.

Voice & Audio Facebook AIwav2vec 2.0Speech Recognition

June 17, 2020 Medium

Image GPT: generative pretraining for images

OpenAI introduces Image GPT (iGPT), a transformer that treats pixels as tokens and shows that GPT-style sequential generative pretraining works on images too, reaching 96.3% on CIFAR-10 with a linear probe.

Multimodal AI OpenAIImage GPTGenerative Pretraining

June 11, 2020 Landmark

OpenAI launches the GPT-3 API in private beta

Two weeks after the paper, OpenAI opens its first API in private beta, a general "text in, text out" interface on top of GPT-3 models, free for the first two months and with waitlist access.

Foundation Models OpenAIGPT-3API

May 2020

May 28, 2020 Landmark

GPT-3: the paper that opens the scaling-laws era

OpenAI publishes 'Language Models are Few-Shot Learners' and shows that at 175B parameters a model learns new tasks from a handful of examples in the prompt.

Foundation Models OpenAIGPT-3Few-shot Learning

May 22, 2020 Landmark

RAG: Retrieval-Augmented Generation enters the literature

Lewis et al. (Facebook AI Research, UCL, NYU) publish the RAG paper, combining a dense retriever (DPR) with a seq2seq generator (BART) for knowledge-intensive tasks, without relying on the model weights for every fact.

Foundation Models Facebook AIRAGRetrieval-Augmented Generation

May 14, 2020 Landmark

NVIDIA A100: Ampere arrives, the AI data-center GPU

At GTC 2020 (a pre-recorded keynote) Jensen Huang announces the A100 GPU, the first on the Ampere architecture: 54 billion transistors, 40 GB HBM2, Tensor Cores with TF32, 2:4 structured sparsity and Multi-Instance GPU with up to 7 instances.

AI Infrastructure NVIDIAA100Ampere

April 2020

April 30, 2020 Medium

OpenAI Jukebox: generating whole songs with vocals

OpenAI releases Jukebox, a generative model that produces raw songs (audio + vocals + lyrics) conditioned on artist and genre, built on a stack of VQ-VAE and autoregressive transformers.

Voice & Audio OpenAIJukeboxMusic Generation

April 10, 2020 Medium

Longformer: sliding-window attention for long documents

Allen Institute for AI publishes Longformer, a transformer that combines local sliding-window attention with global attention on special tokens, scales linearly with length, is pre-trained on 4096-token sequences, and beats RoBERTa on long-document tasks.

Foundation Models AllenAILongformerLong Context

March 2020

March 10, 2020 Medium

ELECTRA: more efficient NLP pre-training than BERT

On March 10, 2020 Google Research introduced and open-sourced ELECTRA (Clark, Luong, Le, Manning; paper at ICLR 2020): instead of masked language modeling, the model learns to detect tokens replaced by a small generator, matching RoBERTa and XLNet on GLUE with less than a quarter of the compute.

Foundation Models GoogleStanfordELECTRA

February 2020

February 13, 2020 Medium

Microsoft Turing-NLG: 17B parameters and the birth of DeepSpeed

Microsoft Research unveils Turing-NLG, the largest announced language model to date (17B), trained with the DeepSpeed library and the ZeRO optimizer, which reduce the memory needed per GPU.

Foundation Models MicrosoftTuring-NLGLarge Language Models

January 2020

January 28, 2020 Medium

Google Meena: the 2.6B end-to-end chatbot

Google introduces Meena, a 2.6B-parameter conversational model trained on 341 GB of social dialogue, along with SSA, a new metric for evaluating chatbot quality.

Foundation Models GoogleMeenaDialogue

January 13, 2020 Medium

Reformer: the transformer that handles very long sequences

Google Research presents Reformer, a transformer variant using LSH attention and reversible layers to go from O(L²) to O(L log L); Google's blog cites context windows up to 1 million words on a single 16 GB accelerator.

Foundation Models GoogleReformerEfficient Transformers

2019

November 2019

November 18, 2019 Medium

Bing serves BERT models on Azure GPUs for every search

Microsoft explains how Bing applies BERT-style transformer models to every query worldwide using over 2,000 Azure GPU VMs that serve more than 1 million inferences per second, which Bing calls its largest improvement in search experience.

AI Infrastructure MicrosoftBingBERT