Overview & Capability
Groundbreaking capability allowing Claude to look at a computer screen, move the mouse cursor, click buttons, and type text to automate complex desktop software workflows.
Key Highlights
- Interacts with legacy desktop apps that lack developer APIs.
- Automates multi-step form filling, software testing, and browser workflows.
- Uses screen coordinate reasoning and visual verification before actions.
Robbie's Independent Take
Verified: 2026-10-11
Best evaluated within a bounded workspace using version control. Configure strict tool approvals and MCP permission gates before granting terminal or file execution access. Monitor token spend carefully when running multi-turn autonomous refactoring loops.
Alternatives & Replaces
- Robotic Process Automation (RPA) licenses ($500+/mo)