Features
Crawl monitoring
We deliver plan-based, scheduled crawling with honest progress, full attempt history and strict safety limits enforced on every request.
Plan-based scheduling
We base your crawl frequency on your plan and Project configuration. Scheduled work is planned in crawl windows rather than presented as 24/7 or real-time crawling.
Honest planning progress
While discovery is still finding new URLs, the final total remains unknown, so CrawlDrift never shows a misleading percentage. You see what is planned, running, completed and failed.
Critical URLs and effective host policy
Prioritize important URLs within the planner while applying the effective host policy, per-host concurrency and politeness limits that protect the origin.
FULL and VERIFICATION crawls
A FULL crawl evaluates the project according to its plan and configuration. A VERIFICATION crawl targets affected URLs only and does not cancel an existing FULL crawl.
Every attempt on record
We record DNS, connect, TLS, time to first byte, download and parse timings for every attempt, alongside status codes, response sizes and redirect hops.
Safe by default
Every fetch and every redirect hop passes scheme, DNS, private-network, size and timeout checks. The crawler respects robots.txt and per-host politeness limits.
Quota-aware behavior
When you exhaust your monthly allowance, we safely complete reserved crawl work, pause new crawls and keep quota-excluded URLs visible in your dashboard. Scheduled work can be skipped when quota is unavailable.
Manual replacement warning
A manual crawl does not silently replace a scheduled or deployment-linked run. The dashboard keeps run type, planning state and quota context visible.
Start monitoring your technical SEO today
Set up your first project in minutes. Verify your domain, start a crawl, and see exactly what changed without the noise.
No credit card required. Starter plan limits apply.