Enabling Native Computer Use for Agents
Listen to the summary
Uses a voice available on your device
Audio options
On this page
Key Takeaways
- Achieves an 86.2 score on the OSWorld-Verified benchmark, scaling to 87.6 with the Max variant.
- Utilizes a massive cloud rollout fleet of nearly 100,000 vCPUs to generate 40,000 verifiable training tasks.
- Employs chunked screenshot folding to maintain visual context by keeping 20 active screenshots while replacing older ones with text markers.
- Improves security by reducing the attack success rate on the RedTeamCUA benchmark from 36.6 down to 16.4 compared to the baseline.
- Uses iterative training loops where reinforcement learning policies are continuously used to refresh data mixtures.
Summary & Methodology Analysis
Qwen-CUA shifts the paradigm for computer-use agents by discarding DOM trees and accessibility layers in favor of a purely visual interface. The agent processes raw screenshots and emits low-level keyboard and mouse events. To handle the challenges of long-horizon tasks, the researchers implemented chunked screenshot folding, which maintains a sliding window of 20 active screenshots while swapping older frames for textual placeholders to manage memory constraints without losing historical context. Training relies on Soft Adaptive Policy Optimization, a reinforcement learning method that optimizes policy sequences, applied over massive parallel rollouts via a fleet of 100,000 vCPUs to generate 40,000 verifiable tasks.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary contribution of this paper?
The paper presents Qwen-CUA, an agent capable of interacting with standard desktop and web software using only pixel-grounded mouse and keyboard inputs.
Q2. Does this agent require special access to the software it controls?
No, it explicitly avoids dependencies on DOM trees, accessibility metadata, or task-specific APIs.
Q3. How does the agent perform on standard benchmarks?
Qwen-CUA reaches 86.2 on OSWorld-Verified, while Qwen-CUA-Max reaches 87.6 on the same benchmark.
Q4. How is visual context managed for long sequences?
The system uses chunked screenshot folding, which keeps 20 active visual frames while converting older ones into fixed textual placeholders.
Q5. What reinforcement learning technique is utilized?
The paper uses Soft Adaptive Policy Optimization, a reinforcement learning method that optimizes decision policies over long action-reasoning sequences.
Q6. How are the training datasets generated?
The team uses a cloud rollout fleet with approximately 100,000 vCPUs to construct about 40,000 verifiable tasks.
Q7. Does the system guarantee complete safety against exploits?
No, while the attack success rate on RedTeamCUA dropped from 36.6 to 16.4, the system retains a residual risk of attack success.
Q8. Are there known limitations in tool usage?
Yes, the agent exhibits a performance trade-off when using Bash tools for navigation, suggesting it has not yet mastered when to switch between modalities.
Q9. What models and benchmarks were used in the evaluation?
Evaluations included Qwen-CUA and Qwen-CUA-Max against baselines like Qwen3.7, GPT-5.5, and Claude Opus 4.8 using benchmarks including OSWorld-Verified, OSWorld 2.0, WebArena, and RedTeamCUA.