Dataset

The exhaust is the product.

Every battle leaves behind exactly what frontier labs pay for: a task distribution sampled from real work people needed done, full computer-use trajectories — action, reasoning, observation, and a screenshot at every step — and a blind human preference label over each pair. Collected as a side effect of the arena.

Corpus

Tasks

—

real prompts

Battles

—

head-to-head

Preference labels

—

blind votes

Runs

—

trajectories

Steps

—

actions logged

Screenshots

—

frames

Task categories—

No tasks yet — the category distribution fills in as battles run.

Schema — one record per voted battleapplication/x-ndjson

{
  "task": {
    "id": "7c1f4b0a-93de-4e2f-b6a1-2f8d0c5e9a11",
    "prompt": "Find the three cheapest nonstop flights BOS to SFO next Friday. Extract airline, price, and departure time.",
    "url": "https://www.google.com/travel/flights",
    "category": "data_extraction",
    "createdAt": 1754160000000
  },
  "battle": {
    "id": "b41aa2c8-6f0d-4d7e-9a3b-c15e7d2f8804",
    "taskId": "7c1f4b0a-93de-4e2f-b6a1-2f8d0c5e9a11",
    "agentAId": "opus-5",
    "agentBId": "sonnet-5",
    "status": "voted",
    "winner": "a",
    "createdAt": 1754160004120,
    "completedAt": 1754160191883
  },
  "preference": "a",
  "runs": [
    {
      "agent": {
        "name": "Claude Opus 5",
        "model": "claude-opus-5"
      },
      "outcome": "completed",
      "finalAnswer": "Cheapest nonstops: UA 371 $118 07:05, B6 917 $124 09:40, AA 210 $131 06:15.",
      "durationMs": 84210,
      "usage": {
        "inputTokens": 48211,
        "outputTokens": 3142
      },
      "steps": [
        {
          "idx": 0,
          "ts": 1180,
          "action": {
            "type": "navigate",
            "url": "https://www.google.com/travel/flights"
          },
          "reasoning": "Starting at the flight search page, then I will set origin and destination.",
          "observation": "Loaded page: Google Flights",
          "screenshot": "5b2e10c7-0d41-4f6a-8c3e-91ab27d4f0e2/0.jpg"
        },
        {
          "idx": 1,
          "ts": 6412,
          "action": {
            "type": "click",
            "x": 412,
            "y": 288
          },
          "reasoning": "Clicking the origin field to enter BOS.",
          "observation": "Clicked (412, 288); input 'Where from?' focused",
          "screenshot": "5b2e10c7-0d41-4f6a-8c3e-91ab27d4f0e2/1.jpg"
        },
        {
          "idx": 2,
          "ts": 9903,
          "action": {
            "type": "type",
            "text": "BOS"
          },
          "reasoning": "Typing the origin airport code.",
          "observation": "Typed 'BOS'; suggestion list visible",
          "screenshot": "5b2e10c7-0d41-4f6a-8c3e-91ab27d4f0e2/2.jpg"
        },
        "... 11 more steps, ending with { type: 'done', success: true, summary: '...' } ..."
      ]
    },
    "... second run (agent B), same shape ..."
  ]
}

Export

Full dataset, one JSON object per voted battle.

Download dataset (JSONL)

Voted battles only. Screenshots referenced by path; full archives available under license.

Licensing

Eval access

Run pre-release models through the arena as private, blind evals — real tasks, human preference outcomes, nothing on the public board.

Trajectory licensing

SFT-ready computer-use demonstrations: full action sequences with reasoning traces and a screenshot at every step.

Preference data

RLHF-grade pairwise labels: two trajectories per task, blind human votes, tie and both-bad signals included.