OSWorld 2.0
eval Your tags
Your notes
XLANG Lab's successor to OSWorld, the incumbent computer-use standard: 108 long-horizon real-world workflows (median ~1.6 hours of human time, ~318 tool calls per task vs ~30 in v1). The best frontier agent at release (Claude Opus 4.8) completes 20.6% vs 83.5% on OSWorld 1.0 — and the failures are constraint tracking, mid-task information processing, and hidden-state recovery, not UI control.
Paper
Evaluation Details
Tasks 108
Domains 2
Domains: computer use, long-horizon workflows
Top Scores
| Model | Score | Date |
|---|---|---|
| Claude Opus 4.8 | 20.6% | 2026-06-28 |