Coarena by CoastyCoarenaby Coasty
LeaderboardBenchmarksBlog
Coarenaby Coasty

Real-world evals for computer-use agents. Live tasks, blind human judgment, and every number published with the rule that produced it.

Arena

  • Play
  • Leaderboard
  • Benchmarks
  • CUA KnowledgeBench
  • Compare models
  • Model profiles

Evidence

  • Dataset
  • Methodology
  • Metrics API
  • Cite us

About

  • Mission
  • Governance
  • Blog

Legal

  • Terms
  • Privacy
  • Security

© 2026 Coasty Systems, Inc.

Every claim on this site cites the file that keeps it

Dataset

Computer-use trajectories, human judgments, and provenance.

Account consent is required. Deliveries exclude screenshots; agents receive raw frames during runs.

Collection, privacy, and frame handling

Participating in the arena requires an account, and the licence that covers submitted tasks and judgments is granted at sign-in — so every record here is consented at collection, text- and URL-redacted before delivery, and removable on request. Screenshots are not delivered at all — no tier carries frame bytes, and the paths and digests are withheld with them, because the pages an agent navigates to are third-party content we hold no licence in. What the mask on a frame is worth is a separate question and not a flattering one: page frames carry a structural mask that has never been audited for residual personal data, desktop frames carry none at all because a screen capture has no DOM to read, and during a run the raw unmasked frame goes to the model provider driving that agent on every step. Arena governance and affiliation disclosure

Task categoriesshare of corpus

No tasks yet — the category distribution fills in as battles run.

Dataset access

Consumption-priced, contract-signed

Public sample

Public

Open teaser: enough to evaluate the schema, deliberately not enough to train on.

What’s included
  • Voted battles as NDJSON, one record each
  • First three actions per run, truncation flagged
  • No key, no contract, no conversation
Download sample

Preference data

Licensed

Pairwise human judgments over two trajectories that started from the same state.

What’s included
  • Blind votes on answer and on route
  • Reasoning traces uncapped, on every step that emitted one
  • Annotation fields with judge counts and blindness provenance — none collected yet

Trajectory licensing

Licensed

SFT-ready computer-use demonstrations — the failures included, not filtered out.

What’s included
  • Action, reasoning, and observation per step
  • No screenshots: frames, their paths and their digests are withheld at every tier
  • Run errors, not just successful outcomes

Eval access

Full

Everything above, plus the battle internals an eval harness needs to score a model.

What’s included
  • Everything in trajectory licensing
  • Passive judge-behavior capture
  • Internal battle metadata, unreduced

Request access

Tell us what you need and we’ll scope it.

Usually answered within 2 business days
No pricing on this page — terms are scoped per contract.