The Website Crawler Insider’s Guide
Everything we know about Website Crawler in one document: what it costs, its grades, getting a refund, getting help and what else to look at. Read it here, or keep a copy.
Website Crawler
The Insider’s Guide
Cloud SEO crawler, data scraper and page monitor priced by pages
Every plan, grade, refund rule and rival among technical SEO tools, read from Website Crawler’s own pages.
Made by WebsiteCrawler.org · checked Oct 23, 2026 · toolgpa.com
Contents
- 01WelcomeWhat it is, who it suits, its grades and the ten things to know first
- 02What it costsEvery plan, the free plan, ways to pay less and how prices have moved
- 03What it doesIts main features, where it runs, what it works with and who makes it
- 04Where it standsIts grades against its category, and whether to keep it or make your own
- 05The alternativesThe tools it is most often weighed against, side by side
- 06Quick answersThe questions people ask about it, each answered in a line or two
- 07About ToolGPAWho made this guide, what else is on the site, and how to cite it
ToolGPA put this guide together from Website Crawler’s own pages, its app-store listings and our grades. Every fact in it is on Website Crawler’s page too, where it is kept up to date. Checked Oct 23, 2026.
Part 01 · Overview
Welcome
What it is, who it suits, its grades and the ten things to know first
2.4
GPA, above 41 of the 83 other technical SEO tools we grade
Website Crawler is a cloud-based crawler that checks a site for more than 100 technical SEO issues, from broken links and canonicals to headings, alt text and thin content. It can render JavaScript, pull custom data with CSS, XPath or JavaScript, monitor pages and uptime, and export results as CSV, JSON, Markdown or PDF. Plans are priced by the number of pages crawled each month.
To get you started, here are 9 things to know about Website Crawler, taken from its own pages and our grades.
Nothing to install, but you create projects, choose HTML or Chrome crawling and add a PageSpeed Insights key for Core Web Vitals.
- 1
Best for cloud crawls priced by the page
Website Crawler is a cloud crawler that checks 100+ technical SEO issues, renders JavaScript on paid plans, scrapes custom data and watches pages and uptime, with an API.
- 2
Paid plans start at $10 a month
As its pricing page puts it: $10 per month for 2,000 pages, billed monthly.
- 3
There is a free plan
Basic plan: 1,000 pages a month, 1 project, the HTML crawler, basic SEO reports, 3 page monitors and API access. Advanced reports show only 5 URLs.
- 4
It runs on Web
Web.
- 5
Technical SEO reports
Finds broken links, redirect chains, canonical errors, duplicate titles and descriptions, heading gaps, missing alt text, thin content and hreflang problems.
- 6
JavaScript rendering
Paid plans can switch from the faster HTML crawler to a Chrome mode that runs each page's JavaScript before checking it, so single-page apps can be crawled.
- 7
Custom data extraction
Scrapes chosen data from pages with CSS selectors, XPath or a JavaScript snippet, exported as CSV, JSON or Markdown, with optional residential proxies.
- 8
A 2.4 GPA, above 41 of 83
That is against the other technical SEO tools we grade, on value, the free plan, features, ease of use and platforms.
- 9
Keep it
Nothing Website Crawler does is plain enough to make in a weekend. The 3 things listed below take a company.
Website Crawler at a glance
- Pricing
- Free plan, from $10 per month for 2,000 pages, billed monthly
- Free plan
- Yes
- Free trial
- None stated
- Runs on
- Web
- Made by
- WebsiteCrawler.org
- Category
- Technical SEO & Site Audits
- Our grade
- 2.4 GPA, fair
The report card
Best for cloud crawls priced by the page
Website Crawler is a cloud crawler that checks 100+ technical SEO issues, renders JavaScript on paid plans, scrapes custom data and watches pages and uptime, with an API. Page credits are shared across projects and lapse each month, free reports on duplicates and Search Console show 5 URLs, and sites behind anti-bot tools need your own proxy.
| Subject | Grade | Comment |
|---|---|---|
| Value for money | B | Starter is $10 a month for 2,000 pages, inside this category's cheaper quarter of $13. Growth is $20 for 10,000 and Scale $60 for 50,000, and yearly billing takes 10% off. |
| Free plan and trial | C | The free Basic plan gives 1,000 pages a month, 1 project, the HTML crawler and the API with no card. But demo reports, such as duplicates, cannibalization and Search Console data, show only 5 URLs. |
| Features | C | 100+ issue checks, JavaScript rendering, CSS, XPath or JavaScript extraction, Core Web Vitals, page and uptime monitoring and crawl comparison. The gap: it can't crawl sites behind anti-bot tools, or blocked where it's hosted, without your own residential proxy. |
| Ease of use | C | No rating on record. Nothing to install, but you create projects, choose HTML or Chrome crawling and add a PageSpeed Insights key for Core Web Vitals. |
| Platforms and integrations | B | Web, with an API and SDKs for Python, Go, JavaScript, Rust, PHP and Java, plus Search Console and PageSpeed Insights. No desktop or phone app. |
An A is worth 4 points, a B 3, a C 2, a D 1 and an F nothing. The GPA is the average of the five.
Who Website Crawler is for
- Fits if your site needs a crawler for a few dollars a month
- Works for developers pulling crawl data into their own code through the API
Who it is not for
- Tight if you stay free, as demo reports such as duplicates and Search Console show 5 URLs
- Watch out: page credits are shared by every project and expire at the end of each month
- Skip it if your site sits behind anti-bot tools, unless you add your own residential proxy
- Tight if you need JavaScript rendering, which the free plan doesn't include
Good to know
Good for
- Small sites that need a crawler for a few dollars a month
- Developers pulling crawl data into their own code through the API
- Agencies sending white-label PDF audits
Worth knowing
- On the free plan, reports marked demo, such as duplicate content, cannibalization and Search Console data, show only 5 URLs.
- Page credits are shared by all projects in the account and expire at the end of each month.
- It cannot crawl sites behind anti-bot tools or blocked where it is hosted unless you add your own residential proxy.
- Refunds are offered within 14 days of buying a plan, on request by email with a reason.
Part 02 · Pricing
What it costs
Every plan, the free plan, ways to pay less and how prices have moved
$10
A month to start. There is a free plan too
Website Crawler has a free plan. Paid plans start at $10 per month for 2,000 pages, billed monthly.
Basic
$0
1,000 pages a month and 1 project with the HTML crawler, data extraction, API access and 3 page monitors. No card required.
Starter
$10
per month for 2,000 pages, billed monthly
3 projects and 2 users, JavaScript rendering, all reports, Core Web Vitals, white-label PDFs, 5-minute uptime checks and 20 page monitors.
Growth
$20
per month for 10,000 pages, billed monthly
10 projects and 4 users, 3-minute uptime checks, 100 page monitors and priority support.
Scale
$60
per month for 50,000 pages, billed monthly
50 projects and 10 users, 1-minute uptime checks, 500 page monitors and priority support.
Enterprise
Custom
Up to 1 million pages a month and 100 projects, unlimited team members and the fastest crawl speed. The page puts it at around $90 a month and up.
The free plan
Basic plan: 1,000 pages a month, 1 project, the HTML crawler, basic SEO reports, 3 page monitors and API access. Advanced reports show only 5 URLs.
Read from websitecrawler.org on Oct 23, 2026.
Ways to pay less
Annual discount
10% off with annual billing
Website Crawler's pricing page shows a 10% discount on the annual plans, which are paid yearly and renew each year.
Stated on websitecrawler.org
Part 03 · Features
What it does
Its main features, where it runs, what it works with and who makes it
1
Place it runs: Web
Cloud SEO crawler, data scraper and page monitor priced by pages. Here is what it does, as its own site describes it, and what it covers: public API, site audit, page speed.
The gap: it can't crawl sites behind anti-bot tools, or blocked where it's hosted, without your own residential proxy.
- Technical SEO reports
- Finds broken links, redirect chains, canonical errors, duplicate titles and descriptions, heading gaps, missing alt text, thin content and hreflang problems.
- JavaScript rendering
- Paid plans can switch from the faster HTML crawler to a Chrome mode that runs each page's JavaScript before checking it, so single-page apps can be crawled.
- Custom data extraction
- Scrapes chosen data from pages with CSS selectors, XPath or a JavaScript snippet, exported as CSV, JSON or Markdown, with optional residential proxies.
- Page speed and rendering
- Paid plans add Core Web Vitals through a PageSpeed Insights key, console errors, render-blocking resources, CSS and JavaScript coverage, and screenshots.
- Monitoring and scheduled crawls
- Watches pages for changes and keeps up to 10 snapshots, alerts on downtime and SSL expiry, schedules crawls and compares two crawls side by side.
- API and Search Console
- An API with SDKs for Python, Go, JavaScript, Rust, PHP and Java starts crawls and returns the data as JSON; Search Console data can be filtered by keyword.
Where it runs
- Web
What it works with
- Google Search Console
- PageSpeed Insights
Website Crawler is made by WebsiteCrawler.org.
Part 04 · Standing
Where it stands
Its grades against its category, and whether to keep it or make your own
41/83
Other technical SEO tools it is graded above
Website Crawler's GPA of 2.4 is higher than 41 of the 83 other technical SEO tools we grade, and level with 14.
It stands best on value for money (B, above 43 of 83) and worst on ease of use (C, above 7).
Its paid plans start at $10 a month, against a middle price of $22.41 for the category: 9 of the 52 others that charge start lower.
On features, ease of use and platforms it scores 2.3 out of 4. 5 tools score higher and cost less.
Each grade against the category
Every bar is all 84 technical SEO tools we grade, cut by the letter each got. Website Crawler’s own letter is outlined.
- BValue for moneyAbove 43 of 83
- CFree plan and trialAbove 18 of 83
- CFeaturesAbove 10 of 83
- CEase of useAbove 7 of 83
- BPlatforms and integrationsAbove 27 of 83
Price against what you get
Each dot is one of 73 technical SEO tools. Further left costs less; higher up scores better on features, ease of use and platforms. Website Crawler is the marked dot.
What paid plans start at, a month (each step along is roughly double) →
Keep it, or make your own?
Whether this is something you could build for yourself with an AI app builder, and what you would give up.
Keep it
Nothing Website Crawler does is plain enough to make in a weekend. The 3 things listed below take a company.
It costs $120 a year.
What you could not
- A crawler that checks every page of a site.
- Speed tests run from real browsers on a schedule.
- It plugs into other products, Google Search Console and PageSpeed Insights among them, and has an API.
What this rests on. Website Crawler’s listing: what it does, where it runs, what it plugs into and what it costs. It is a judgement about the kind of software this is. We have not built a replacement for it, and making your own means looking after it too. How we decide, and every tool’s verdict.
Part 05 · Alternatives
The alternatives
The tools it is most often weighed against, side by side
Screaming Frog SEO Spider, Sitebulb, SiteGuru, JetOctopus and CrawlCenter
The 5 tools Website Crawler is most often weighed against, with where each one differs from it.
- 2.4GPA
Screaming Frog SEO Spider
Free plan · from $279/yr
Desktop website crawler for technical SEO audits.
- Only Website Crawler lists a web app.
- Only Screaming Frog SEO Spider lists Windows, macOS and Linux apps.
- Website Crawler also covers page speed.
- 2.6GPA
Sitebulb
Free trial · from $18/mo
Website auditing crawler for Windows and Mac, with a cloud edition.
- Website Crawler has a free plan. Sitebulb does not, though it has a free trial.
- Only Sitebulb lists Windows and macOS apps.
- Website Crawler also covers page speed.
- 2.2GPA
SiteGuru
Free plan · from $39/mo
SEO audit tool that turns site crawls into a prioritized task list.
- SiteGuru also covers reports, backlink analysis, schema markup and internal linking.
- 2.2GPA
JetOctopus
Free trial · from $349/mo
Cloud SEO crawler with log analysis, GSC data and AI bot tracking.
- Website Crawler has a free plan. JetOctopus does not, though it has a free trial.
- Website Crawler also covers page speed.
- JetOctopus also covers AI search visibility, internal linking and log file analysis.
- 2.4GPA
CrawlCenter
Free plan · from $3/mo
Simple cloud crawler with on-page SEO reports and a free plan.
- Website Crawler also covers page speed.
- CrawlCenter also covers schema markup.
Part 06 · Questions
Quick answers
The questions people ask about it, each answered in a line or two
14
Questions, each answered in a line or two
The short version of everything above, for when you only need the answer. Each one is told in full in its part of the guide.
Is Website Crawler free?
Website Crawler has a free plan. Basic plan: 1,000 pages a month, 1 project, the HTML crawler, basic SEO reports, 3 page monitors and API access.
How much does Website Crawler cost?
Website Crawler lists 5 plans. Basic: $0.
What platforms does Website Crawler work on?
Website Crawler is available on Web.
Does Website Crawler offer any discounts?
Website Crawler publicly lists the following: 10% off with annual billing.
Does Website Crawler have an Android app?
No Android app was listed when its pages were read on Oct 23, 2026. Website Crawler is listed for Web.
Does Website Crawler have an iPhone app?
No iPhone app was listed when its pages were read on Oct 23, 2026. Website Crawler is listed for Web.
Is Website Crawler open source?
Website Crawler is not described as open source on its own site, as read on Oct 23, 2026.
Does Website Crawler have an API?
Yes. Website Crawler offers a public API, according to its own site.
What does Website Crawler integrate with?
Website Crawler's listing names Google Search Console and PageSpeed Insights.
Who makes Website Crawler?
Website Crawler is made by WebsiteCrawler.org.
What are the alternatives to Website Crawler?
The closest alternatives to Website Crawler in our directory are Screaming Frog SEO Spider, Sitebulb, SiteGuru, JetOctopus and CrawlCenter.
How does Website Crawler compare with other technical SEO tools?
Website Crawler's GPA of 2.4 is higher than 41 of the 83 other technical SEO tools we grade, and level with 14.
Is Website Crawler good value for money?
ToolGPA grades Website Crawler B for value for money. Its paid plans start at $10 a month, against a middle price of $22.41 for the category: 9 of the 52 others that charge start lower.
Could I make my own version of Website Crawler?
Nothing Website Crawler does is plain enough to make in a weekend. The 3 things listed below take a company.
Where this guide comes from
The prices, plans, platforms and features were read from Website Crawler’s own pages on Oct 23, 2026, and the grades were given on Oct 23, 2026. Graded from this listing, whose facts were read from Website Crawler's own pages on 23 October 2026. Not based on using the product, and shorter than our full reviews.
Every tool we list gets five grades, A to F, each with its reason, and a GPA that averages them. How each is decided is at toolgpa.com/grades. This guide is ToolGPA’s own work; it is not made, paid for or approved by WebsiteCrawler.org. Prices change, so check websitecrawler.org before you buy.
Part 07 · About
About ToolGPA
Who made this guide, what else is on the site, and how to cite it
3,189
Tools graded, in 94 categories
ToolGPA grades software the way a school report grades a pupil: five subjects, a letter for each with the reason written beside it, and a GPA that averages them. Prices, plans and platforms are read from each maker’s own pages and dated, so you always know how fresh a figure is.
Every grade, review and ranking on ToolGPA is our own judgement. No company can pay for a grade or change one. Every tool on the site has a guide like this one, free to download and pass on. It all lives at toolgpa.com.
3,189
tools graded
94
categories
426
price changes on record
579
apps priced in 12 countries
What else is on toolgpa.com
- Website Crawler, kept up to dateEverything in this guide, with prices and grades as they are todaytoolgpa.com/tools/website-crawler
- How grades are givenThe rules behind every letter and every GPAtoolgpa.com/grades
- Value chartsWhat each tool costs against what it scores, for every categorytoolgpa.com/charts
- CompareAny two tools side by side, plan by plantoolgpa.com/compare
- Price changesEvery price rise and cut on record, each with its sourcetoolgpa.com/price-changes
- Prices by countryWhat apps cost on the App Store in 12 countriestoolgpa.com/prices-by-country
- Pay lessEvery public discount: yearly billing, students, nonprofitstoolgpa.com/save
- Keep or makeWhich tools are worth paying for, and which you could build yourselftoolgpa.com/keep-or-make
Scan for the live page
Prices and grades as they are today
Quoting this guide?
Please name ToolGPA and link to the live page, so readers can check the figures for themselves:
ToolGPA, "Website Crawler: The Insider's Guide", checked Oct 23, 2026. https://toolgpa.com/tools/website-crawler
Know someone choosing between technical SEO tools?
Send them this guide: toolgpa.com/tools/website-crawler/guide
ToolGPA elsewhere: youtube.com/@toolgpa
See our other guides
The best-graded technical SEO tools after Website Crawler, each with an Insider's Guide of its own.
- Technical SEO & Site Audits Insider's GuideEvery one of the 84 technical SEO tools we grade, compared
- Lighthouse Insider's GuideGoogle's open-source audit of page speed, accessibility and SEO3.2 GPA
- PageSpeed Insights Insider's GuideFree Google speed test with real-user Core Web Vitals and Lighthouse3.2 GPA
- Yellow Lab Tools Insider's GuideFree open-source audit of a page's speed and front-end code3.2 GPA
Choosing software? Every tool graded, every price dated.
Compare tools at toolgpa.com ›Everything in this guide is ToolGPA’s own summary of what Website Crawler’s pages, its app-store listings and public reviews said on the dates shown. Grades are our judgement under the rules at toolgpa.com/grades. Every grade, review and ranking on ToolGPA is our own judgement. No company can pay for a grade or change one. Website Crawler is a trademark of its owner, and this guide is not made, paid for or approved by WebsiteCrawler.org. Prices and plans change: check websitecrawler.org before you buy.