XLANG Lab's successor to OSWorld, the incumbent computer-use standard: 108 long-horizon real-world workflows (median ~1.6 hours of human time, ~318 tool calls per task vs ~30 in v1). The best frontier agent at release (Claude Opus 4.8) completes 20.6% vs 83.5% on OSWorld 1.0 — and the failures are constraint tracking, mid-task information processing, and hidden-state recovery, not UI control.

Paper

Evaluation Details

Tasks 108
Domains 2
Domains: computer use, long-horizon workflows

Top Scores

Model Score Date
Claude Opus 4.8 20.6% 2026-06-28
evalagentscomputer-use

Related