A big crawl doesn't fit in one server window, so audits checkpoint: around the 4-minute mark, a leg saves its progress — which pages are done, and what they contained — and a fresh leg resumes exactly there. Up to 5 legs chain automatically, which covers roughly 20 minutes of crawling and the full 500-page ceiling on ordinary hosts.
You'll see the run report Resuming crawl with the pages already done. Credits charge once for the completed audit, not per leg.
There's nothing to turn on — resuming is automatic. This page matters when a site is big and slow, because each leg spends its window on fewer pages and the 5-leg budget can run out.
Run the audit
To audit a large site:
- Run the audit normally and watch the progress. Steady page counts across legs means it'll finish.
- If a run dies anyway, wait a few minutes. A background sweep picks up the checkpoint and continues the crawl on the server, even with the tab closed. Re-running the audit yourself starts a fresh crawl.
- For chronic cases, work through the fixes below.
Read the finished report
A finished multi-leg audit is a normal report — the health strip's crawl count tells you your coverage. A run that exhausted its legs reports what it crawled. A partial crawl of a giant site is still a valid sample, and template-level problems repeat, so the findings usually stand.
Limits and usage
- The scope ceiling is 500 pages for an audit started through the API or the MCP connector, and the in-app depth picker goes up to 200 pages (why →). Legs exist to reach the ceiling, not to exceed it.
- Slow hosts consume the window on latency. A site serving multi-megabyte pages at several seconds each crawls fewer pages per leg — that cost is real, and it's also what your visitors pay.
- Site builders that throttle rapid requests slow legs further. The crawler backs off rather than fighting.
If something isn't working
Runs exhaust all 5 legs on a mid-size site
Why this happens: the host is slow — the site isn't actually big.
To fix: lower the scope for a faster complete picture, and treat the slowness itself as a finding. It's a ranking factor.
Progress resets instead of resuming
Why this happens: a manual re-run always starts a new audit — only the automatic legs and the background sweep read the checkpoint.
To fix: give a dead run a few minutes to resume on its own before re-running.
Heavy pages time out one by one
Galleries and portfolios are the usual case. These pages are skipped after their per-page budget and named in the report rather than sinking the run. Fixing their weight helps your visitors and future crawls alike.
How long until the background sweep picks up my dead run?
Up to about 15 minutes. The sweep runs every 10 minutes and only claims a run whose checkpoint has sat idle for 6 minutes — deliberately, so it never steals a leg that's still crawling.
When does a stalled run stop trying for good?
After 30 minutes with no progress it's marked dead. The sweep also stops resuming a checkpoint after 3 attempts that produce nothing new — a run that keeps advancing earns fresh attempts.