OpenCUA: Open Foundations for Computer-Use Agents

Vision-language models have demonstrated impressive capabilities as computer-use agents (CUAs) capable of automating diverse computer tasks. As their commercial potential grows, critical details of the most capable CUA systems remain closed. As these agents will increasingly mediate digital interactions and execute consequential decisions on our behalf, the research community needs access to open CUA frameworks to study their capabilities, limitations, and risks. To bridge this gap, we propose OpenCUA, a comprehensive open-source framework for scaling CUA data and foundation models. Our framework consists of: (1) an annotation infrastructure that seamlessly captures human computer-use demonstrations; (2) AgentNet, the first large-scale computer-use task dataset spanning 3 operating systems and 200+ applications and websites; (3) a scalable pipeline that transforms demonstrations into state–action pairs with reflective long Chain-of-Thought reasoning that sustain robust performance gains as data scales. Our end-to-end agent models demonstrate strong performance across CUA benchmarks. In particular, OpenCUA-32B achieves an average success rate of 32.5% on OSWorld-Verified, establishing a new state-of-the-art (SOTA) among open-source models and surpassing OpenAI CUA (GPT-4o). Further analysis confirms that our approach generalizes well across domains and benefits significantly from increased test-time computation. We release our annotation tool, datasets, code, and models to build open foundations for further CUA research.

AgentNetTool: The AgentNet Tool is a cross-platform annotation application that captures user interactions across Windows, macOS, and Ubuntu. It records screen videos, mouse/keyboard events, and metadata, enabling scalable collection of real-world computer-use demonstrations.
AgentNet Method: Raw user demonstrations are processed into clean, learning-ready state–action trajectories. The resulting trajectories include inner monologue-style thoughts and action history, making them suitable for vision-language model training.
AgentNet Dataset and AgentNetBench: The processed data is curated into the AgentNet Dataset and AgentNetBench. The dataset covers diverse open-domain tasks across 100+ applications and 200+ websites. The benchmark provides task instructions, step histories, and multiple gold-standard actions per step for efficient offline evaluation.
OpenCUA Models: OpenCUA agent models are trained on the dataset using reflective Chain-of-Thought reasoning, multi-image histories, and mixed-domain data. They can execute in realistic desktop environments across operating systems to perform computer-use tasks.

Efficient and accurate annotation is essential for collecting high-quality computer-use agent data, yet no existing tools support natural, cross-platform task recording by non-technical users. To address this, we developed a user-friendly annotation tool that streamlines the collection and verification of computer-use demonstrations, runs on annotators' personal computers and records demonstrations in the background, capturing: (1) screen videos, (2) mouse and keyboard signals, and (3) accessibility trees (Axtree).

We conduct supervised fine-tuning (SFT) on Kimi-VL-A3B, Qwen2-VL-7B, Qwen2.5-VL-7B and Qwen2.5-VL-32B, and obtained our OpenCUA model variants: OpenCUA-A3B, OpenCUA-Qwen2-7B, OpenCUA-7B, and OpenCUA-2.5-32B. Our best models - OpenCUA-7B and OpenCUA-32B - achieves superior performance in agent grounding and planning benchmarks.

We appreciate Su Yu, Caiming Xiong, Binyuan Hui for their helpful feedback and discussion on this work. We appreciate Zaida Zhou, Zhengtao Wang, Flood Sung, Hao Hu, Huarong Chen, Calvin, Qizheng Gu and Dikang Du from Kimi Team for the great infrastructure provided and helpful discussion. We sincerely thank our all the annotators for their great effort on this project.