Benchmarks
Published
Round 0: Playwright vs Puppeteer
10 scripted tasks on public practice sites, 10 runs per task per library, same Chromium binary. Run 2026-11-21. Raw JSON and scripts included.
In preparation: Round 1 — AI browser agents
Not run yet. Planned design, so you can critique it before we publish numbers:
- Candidates: Browser Use, Stagehand, Skyvern, and a hand-written Playwright script as the non-AI baseline for each task.
- Tasks: a fixed set of 30–50 tasks on public practice and sandbox sites — form filling, multi-step search, table extraction, pagination, logged-in flows on demo accounts. No tasks against sites whose terms forbid automation.
- Metrics per task: success rate over repeated runs, wall time, number of model calls/steps, model cost (from token counts and the provider's list price on the run date), and failure category.
- Controls: the same model for every framework where the framework allows it; versions pinned; full logs and traces published.
Want a tool or task included? See about for how to reach us.