Can search crawl and find your pages?
Before search can rank a page, it has to be allowed to crawl it and able to find it. Two files decide that: robots.txt and your sitemap. This reads both and tells you what is blocking or missing.
Why a good site still does not get crawled
The content can be right and the pages can still never be seen, because the plumbing that lets search in is broken or missing. It is invisible until you check it.
One line can hide your whole site
A stray Disallow: / in robots.txt tells every search engine to crawl nothing. The site can be perfect and still be invisible.
No sitemap, slower discovery
Without a sitemap, search finds pages only by following links. New and unlinked pages can sit undiscovered for weeks.
A broken sitemap helps no one
A sitemap that does not parse is thrown away whole, so every page in it loses the discovery boost and nobody tells you.
One site in, a fix list out
Paste your site
Your home address. No sign-up, nothing installed.
It reads robots and the sitemap
robots.txt and /sitemap.xml, checked the way a crawler reads them.
You get a ranked fix list
Anything blocking or missing, most costly first, with the exact fix.
What it checks
The gates search passes through before it can crawl and index you, each read the way a crawler reads them and scored by how much it holds you back.
A site-wide block
Whether robots.txt is quietly telling search to crawl nothing, the single most expensive mistake here.
Sitemap discovery
Whether robots.txt points crawlers at your sitemap, the first place they look for it.
The sitemap exists and parses
Whether a sitemap is actually there and is valid XML a search engine can read.
The sitemap lists pages
Whether it holds real URLs or child sitemaps, or parses to nothing and gives search no help.
Straight talk about what it cannot see
It reads the two files a crawler starts from. It is honest about the edges of that.
It checks the usual places
It reads /robots.txt and any Sitemap line in it, not every unusual location a sitemap could be published to.
It counts, it does not crawl
It counts what the sitemap lists. It does not fetch each URL to confirm every one still resolves.
The robots read is basic
It catches a site-wide block and Sitemap lines, not every per-path or per-bot rule a large robots.txt can hold.
It keeps the report, not your site
Your files are read, checked, and thrown away. Only the report is stored, and only for you.
Check the plumbing before you chase rankings
Run it on your site right now. If search cannot crawl you, nothing else you do for SEO matters. Every Growth Tool is included in one subscription: start a free trial and scan any site.
Check my site