SWE-agent: Princeton's open source agent solving 12% of SWE-bench issues
In one sentence On April 2, 2024 Princeton's NLP group releases SWE-agent, an open source agent that with GPT-4 Turbo solves 12.29% of real issues across the full SWE-bench thanks to an agent-computer interface (ACI) designed for language models. The May paper reports 12.5%.
Fixing a bug in a real GitHub repository requires understanding the code, moving between files, editing the right spot and running tests. SWE-agent hands this work to a language model, but instead of giving it a generic terminal it provides an interface designed for it: the ACI (Agent-Computer Interface).
With GPT-4 Turbo, SWE-agent solves about 12% of SWE-bench's 2,294 issues. A few weeks earlier Cognition's Devin had claimed 13.86%, but on a 25% subset of the benchmark; SWE-agent gets a close result on the full set and is open source.
Companies
Princeton University
Tools
SWE-agent, GPT-4 Turbo, SWE-bench
Tags
Sources