Crawlers head to head - scrape, extract, search - same page, same moment, no caching. The open benchmark for crawlers, in preview.
Multi is a live side-by-side comparison. Rankings live on the Leaderboard.
Community votes
blind Battle votes · Elo rating · the crowd decides| # | Crawler | Rating | Record | Votes |
|---|
Arena score
our algorithm · objective, no votes| # | Crawler | Quality | Speed | Cost | Total |
|---|
Two boards, kept separate. Community votes = pure human preference (Elo from blind A/B). Arena score = our objective algorithm. Demo rows are placeholders until keys are added. How the scoring works →
How the Arena scores crawlers
Neutrality is the whole point - we compete in our own arena, so every number is either a published formula anyone can check, or a blind human vote. Here is exactly how it works.
Two boards, never mixed
Crawling is unusual: it has a correct answer (did you extract the real content?), which chat benchmarks don't. So we keep two independent boards and never blend them.
1 · Community votes human preference
In Battle mode two crawlers run on the same case with their names hidden. You pick the better output; names reveal after you vote. Blind voting is what keeps it honest - you can't pick the brand you know.
- Rating is Elo (the chess / LMArena method): everyone starts at 1000, K-factor 24, a win moves rating toward your opponent's strength, a tie is a half-win for both.
- Record is your wins-losses-ties; Votes is the total cast on you.
- One vote per person per matchup (session + IP), rate-limited, over a rolling 30-day window.
2 · Arena score objective algorithm
This board is pure measurement - no votes. Each crawler runs on our own dynamic, uncacheable test pages (we wrote them, so we know the right answer) with caching forced off (a fresh URL every run), and we measure three things:
- Quality = content-F1 against the known main content. Precision (how much of what it returned is real content) and recall (how much of the real content it captured), combined. It rewards capturing the article and penalizes leaking nav/footer boilerplate or dropping text. 100 = perfect.
- Speed = real measured latency, rolling average (the fastest crawler anchors the scale).
- Cost = published per-call price tier (free > $ > $$ > $$$).
Total = 0.60 × Quality + 0.25 × Speed + 0.15 × Cost
Quality dominates because quality is the point; speed and cost break ties. Everything is a rolling 30-day window.
Fairness
- Same page, same moment, cache off - the only difference in a matchup is the crawler.
- We author the ground truth, so quality is objective, not opinion.
- Crawl4AI is just another entry pointing at our public API - the arena calls it exactly like any competitor, with no shortcut.
- The formula and test harness are open - this is a standard, not a scoreboard we control.
Live vs demo
A crawler runs live when its API key is present; until then it shows a clearly-labeled demo placeholder so the board is readable. Adding the key flips it to live with no code change.
Names are hidden until you vote - that is what keeps the ranking honest. Vote recording + the vote-weighted leaderboard land next.
Are you a crawler? Get on the board.
Register your crawler, or bring ideas for what should count. We build the benchmark in the open.
v1 preview - live providers run for real (cache off); demo providers show placeholder output until a key is added.