Devin: Cognition unveils its autonomous 'AI software engineer'
In one sentence Cognition announces Devin, an agent that plans and executes development tasks with a shell, editor and browser in a sandbox. It claims 13.86% on SWE-bench, measured on a random 25% subset (570 of 2,294 tasks).
On March 12, 2024 the startup Cognition unveils Devin, which it calls an "AI software engineer": an agent you hand a development task to and that carries it through on its own, reading code, writing changes, running them and fixing errors.
The examples in the announcement include building and deploying a Game of Life website to Netlify, fixing a bug in the sympy library, fine-tuning a model from a GitHub repository, and a job taken from Upwork.
The number that gets people talking is the SWE-bench result: 13.86% of issues resolved, against a previous best of 1.96%. The figure, however, is not computed on the whole benchmark but on a random sample of a quarter of the tasks. Access to the product is limited at first.
Companies
Cognition
Tools
Devin
Tags
Sources