Cua Releases Open-Source Tools for AI Agent Computer Use and Desktop Automation

Introduction to Cua's AI Agent Tools

Cua has introduced a suite of open-source tools designed to enable AI agents to use computers more effectively. The platform provides capabilities for desktop automation, isolated cloud desktops, and local macOS virtual machines, aiming to facilitate the development of AI agents that can navigate various computing environments.

Core Components and Features

Key components of the Cua platform include Cua Fleets for provisioning Linux desktops and running commands, CUA-S1 for specialized computer-use decision models, and Cua Driver for operating applications like Calculator. Lume allows for the creation of Tahoe VMs with SSH connectivity, while Cua Bench provides tools for creating and verifying simulated tasks for agent evaluation. The platform supports agents interacting with native desktop applications and browsers across macOS, Windows, and and Linux.

Cloud and Local Sandbox Environments

Cua offers isolated cloud desktops through run.cua.ai, where Fleets maintain sandbox capacity for code execution and interaction with applications. The Sandbox SDK is used for running commands, capturing screenshots, and interacting with apps within these environments. Local sandboxes are also supported, sharing the Sandbox SDK but differing in credentials, images, operations, and runtime requirements.

Agent Integration and Operation

Agents can connect to the Cua platform via CLI, MCP, or typed SDKs. The system supports 'Computer-Use 2.0,' where an agent can move between code, APIs, and graphical interfaces within a single task. Background delivery allows agents to operate without moving the pointer or taking focus, depending on app and platform support.