Google Search Console
SEO & Analytics
Google's free report on how your site shows up in search, including AI results.
Screaming Frog SEO Spider
Crawls your whole site the way Google does and lists what's broken.
FreemiumWorks with Claude Code
Visit website What is it?
Screaming Frog is a program you install on your computer that visits every page of a site and reports back: broken links, missing titles, duplicate pages, slow images, redirect chains and anything else that trips Google up.
It's the tool technical SEO people have used for over a decade, and it now connects to AI models to summarise pages or extract information during a crawl. Exports go straight into a spreadsheet, and it can run from the command line, so Claude Code can trigger and read crawls.
It looks like a spreadsheet from 2010 and assumes you know what the columns mean. Non-technical users may prefer the audit built into Ahrefs.
Who it's for
Anyone who needs a thorough technical check of a site, especially before or after a redesign or migration.
Best for
- Finding broken links
- Checking redirects after a migration
- Auditing titles and headings
- Exporting a full page list
Why it made the list
Still the most complete and trusted site crawler, and the free tier covers most small sites.
You might also like
Ahrefs
SEO & Analytics
Research keywords, check backlinks and audit your site in one well-known toolkit.
DataForSEO
SEO & Analytics
Raw search, keyword and backlink data you pay for by the query, built for AI agents.
Microsoft Clarity
SEO & Analytics
Free heatmaps and session replays that show where visitors get stuck.