Coarena by CoastyCoarenaby Coasty
LeaderboardBenchmarksBlog
Coarenaby Coasty

Real-world evals for computer-use agents. Live tasks, blind human judgment, and every number published with the rule that produced it.

Arena

  • Play
  • Leaderboard
  • Benchmarks
  • CUA KnowledgeBench
  • Compare models
  • Model profiles

Evidence

  • Dataset
  • Methodology
  • Metrics API
  • Cite us

About

  • Mission
  • Governance
  • Blog

Legal

  • Terms
  • Privacy
  • Security

© 2026 Coasty Systems, Inc.

Every claim on this site cites the file that keeps it

Security

Implemented controls and known limitations. Security questions, reports and disclosures: founders@coasty.ai.

Architecture

  • Encrypted in transit, every hop. TLS from the client to Cloudflare, Cloudflare to the application on Render, and the application to the database and object store.
  • Encrypted at rest. AES-256 through our managed providers’ storage. The encryption is the provider’s control, not one we implement — stated as such.
  • Agent runs are ephemeral and isolated. Each run executes in a fresh remote sandbox — its own Linux desktop — destroyed when the run ends. Runs share nothing.
  • Agent egress is filtered. A denylist blocks localhost and private address space, and DNS is re-resolved on a short TTL to limit rebinding — a task cannot be steered at internal infrastructure.

Data protection

  • Redaction happens at write time, not at delivery. Text and artifacts pass a PII redactor and a blinding pass before they are stored, so the record at rest is already scrubbed rather than depending on a filter at the edge. Files you upload are the exception and are stored exactly as you sent them: the redactors read text, and a spreadsheet or an archive is not text. They are never published in any dataset tier and are readable only by the account that uploaded them.
  • The object store is private. Screenshots and frames live in a bucket created private, with a size cap and a MIME allowlist. Uploaded files live in a second private bucket with its own larger cap, because widening the frame bucket to admit a 50MB document would have widened it for frames too. There is no public read path and no signed-URL serving; every read is brokered by the application against an authenticated session.
  • Uploads are checked against their own bytes. A file is accepted on what its leading bytes actually are, not on the type your browser declared, against a closed allowlist; an executable is refused whatever it is named. There is no malware scanning — we do not run an engine and will not imply one. What stands in its place is the allowlist, no execution path on our side, per-account quotas, and a retention clock.
  • Raw IP addresses are never persisted. They exist in memory for the life of a request and are hashed before anything is written.
  • The public dataset tier is stripped. No screenshots, no reasoning traces, no judge behaviour — capped and stratified. Richer tiers require a licence.
  • Records are refused with a 404, not a 403. A request outside the authorised path learns nothing — not even that the record exists.

Security properties are tests, not policy

The properties that matter here are encoded as automated suites that run on every change: network egress policy, judging blindness, battle access control, consent integrity, share exposure, rating fairness. A change that would weaken any of them is a failing build, not a discovered incident. The suites are in the repository under verify:* and their claims are published, with the file that keeps each one, at /governance.

Access control

  • Two people hold production access. Both founders, both US-based, both under executed confidentiality agreements. No contractors, no agencies, no crowdworkers.
  • MFA everywhere, no SMS. Authenticator-app one-time codes on every personnel account — identity, source control, hosting, database, storage and sandbox consoles alike.
  • Administrative routes take a separate credential. A bearer token compared in constant time, not derivable from any signed-in session.
  • Endpoints are locked down. Full-disk encryption on every machine; automatic screen lock at 30 seconds of inactivity.
  • Revocation is rotation. Departure means account removal across every console within 24 hours — and mandatory rotation of every shared secret the person could have observed, because removal without rotation does not end access.

What model providers receive

This is the material disclosure. CoArena exists to compare computer-use agents, so runs assigned to a provider’s model send that provider the task prompt verbatim and the raw, unmasked screenshot of every step — that is what a model must see in order to act. Masking applies to what we store and publish, not to inference. This is disclosed to every user at sign-in and documented in the privacy policy.

Incidents

We have had no security breach and no incident requiring notification. If one occurs, affected parties are notified within 24 hours of our becoming aware — initial notice on discovery, not on completed investigation, with written follow-up as facts are established. Report anything you find to founders@coasty.ai; both founders read it, and good-faith research on your own account will never be met with legal threats.

What we do not have yet

Stated with the same weight as everything above, because a posture you cannot audit is a story:

  • No SOC 2, no ISO 27001. We hold no security certification and have no third-party audit report to show. We would rather say that directly than imply a posture we cannot evidence; a SOC 2 readiness process is the committed next step as the company grows.
  • No dedicated security team. Security is owned directly by the founders, with a named accountable owner.
  • No enforced retention schedule, with one exception. Most records are retained indefinitely today. The exceptions enforced in code are raw IPs never stored, rate-limit counters deleted in minutes, sessions expiring at 30 days, and files you upload, which carry a 30-day clock — a document you attached is your property in a way a vote is not. The reasons for the rest are documented in the privacy policy, which marks every unbuilt control as unbuilt.
  • No SIEM. Application, platform and database logs exist, with deduplicated alerting to an internal channel — but no centralised aggregation and no scheduled access reviews yet.

Who processes data on our behalf

13 providers, from the database to the model APIs, each listed with what actually reaches it — including the rows a softer page would omit. The full list, with its change history policy, is at /subprocessors.