Skip to main content
sus.botdocs
  • SEO
  • AI
  • QA
  • Privacy
  • Accessibility
  • Security
  1. Docs
  2. SEO
  3. robots.txt

SEO audit

robots.txt

6 checks · 3 findings · scope: domain

Audit id: seo.robots_txt.

Checks

  • Robots.txt not found not_found
  • Robots.txt has errors syntax_errors
  • Blocks important parts of the site blocks_important_paths
  • Blocks files needed to show pages blocks_resources_needed_for_rendering
  • No sitemap named in robots.txt sitemap_line_missing
  • Rules changed since last check rows_added_or_removed

Findings

  • robots.txt blocks pages you want found, or files the pages need to render
  • Rules were added to or removed from robots.txt since the last fetch
  • robots.txt is missing, cannot be read, or does not name the sitemap
  • All audits
  • About sus.bot
  • Open the app
  • Contact support