SEO finding
A newly blocked path is still in the sitemap or the navigation
SEO Error Severity: High
Part of the audit A newly blocked path that has search traffic. Finding id: seo.blockedPathStillPromoted.
What it costs you
The site tells crawlers to stay away from pages it presents as important. Those pages stop being crawled and lose their place in search results.
What to do
If the block is a mistake, delete or narrow the line. If it is meant, take the URLs out of the sitemap and the navigation.
What it checks
- Blocked path is listed in the sitemap
- Blocked path is linked from the menu
How to fix it
Message for IT
- robots.txt line [line]: Disallow: [path]
- This rule was not in the previous version of the file. [urls] URLs under it are in the sitemap or the navigation, for example [example].
- Change: delete line [line] or replace it with the narrower path that was intended.
How to check: Fetch /robots.txt and confirm the line is gone; run this rule again.
A real case
A development team released a robots.txt that blocked the production site, and the owner lost all organic traffic, 85 to 90% of the site's total. Almost two months later it had not come back.
How to verify by hand
Search Console clicks and impressions for the path before and after the rule changed.
What this relies on
Blocking a path that earns search traffic is a mistake unless the content was removed on purpose.