Computer Use: Claude learns mouse and keyboard
In one sentence Anthropic enables 'Computer Use' on Claude 3.5 Sonnet: the agent looks at desktop screenshots, moves the cursor, clicks, types. For the first time a commercial LLM operates directly on the GUI.
Anthropic ships a new beta feature: Claude can "see" the screen (screenshot) and use it like a human. Moves the mouse, clicks buttons, types text, takes screenshots, repeats.
The idea: instead of building custom APIs for every service, the AI uses GUIs designed for humans. You fill an Excel form, search a website, complete a legacy request without an API → all as if a person were behind it.
Slow, error-prone, with obvious security issues (can click anywhere). But it's the first serious step toward agents operating in the existing software world, not just a sandbox.
Companies
Anthropic
Tools
Claude 3.5 Sonnet (new)
Tags
Sources