Methodology

A computer-use benchmark evaluates how an AI agent interacts with software to carry out tasks. Coarena records agent attempts on user-submitted browser and desktop work and collects blind human comparisons of their results.

There is no single result that establishes the best model for every workload. Coarena’s leaderboard orders the current roster by published human-preference rating. Compare completion, speed, recovery, cost and the direct matchup for your use case, and account for provisional ratings and uncertainty.

The published ranking uses a Bradley-Terry fit over eligible comparisons, with uncertainty clustered by judge. Provisional models use an online Elo fallback. The rating process excludes operator experiments, compromised blindness, harness errors, owner-stopped runs and unequal continuation budgets, and applies label-quality rules.

No. Completion is the share of runs where the agent reported finishing. A preferred answer can still be wrong, and a completed run can still miss a requirement. Verified completion is the checkable subset: on runs whose task carries an executable evaluator, an independent check of the recorded end state confirms or refutes the claim, and the share of runs such a check could decide is printed first. Independent acceptance checks are separate evidence and are not implied by the completion metric.

Recovery is the fraction of failed actions whose next recorded action did not fail. A failed final action has no successor and is excluded from that denominator. This is a next-action proxy, not proof that the overall task recovered successfully.

Estimated inference cost per completion applies the pricing table’s per-token list rates to input and output tokens across all measured runs, including unsuccessful runs, then divides by agent-reported completions. It excludes infrastructure, caching discounts and negotiated rates and is not a historical billing total.

Model and comparison pages publish aggregate benchmark metrics. Individual tasks, prompts, answers, recordings and evaluation records are not included. Result sharing is a separate choice made by the person who submitted a battle.

Yes. Cite the specific model or comparison page and its last-updated timestamp. The public benchmark JSON contains aggregate model metrics and matchup rates. Private task prompts, recordings and the commercial trajectory dataset are not included in that public download.