aeo.robots-txt-unreachable

What to change

A finding of this rule on a page carries the exact edits for that page: the markup it quotes, and what to write instead.

When it runs

Runs on every AEO audit: of a page, a crawl, a build directory or HTML. A warning is a problem that costs a page ranking, clicks or citations. Each finding takes 5 points off the page's score.

The severity here is the rule's usual one: a finding can be raised or lowered when the page gives a reason.

Run it, or skip it

Check a page for this rule alone, after a fix for instance:

npx @rivi-software/agentic-seo-check aeo https://example.com --rule aeo.robots-txt-unreachable

Or from an assistant, with the MCP tool seo_aeo_audit:

{
  "url": "https://example.com",
  "rules": [
    "aeo.robots-txt-unreachable"
  ]
}

To leave it out of an audit, pass --disable aeo.robots-txt-unreachable on the command, or disableRules: ["aeo.robots-txt-unreachable"] to a tool.

• Warningaeo.article-dates-missingArticle structured data lacks datePublished (warning) or dateModified (info), which tell answer engines how fresh the content is.
• Infoaeo.article-date-not-visibleThe article's structured data has a date, but the page shows none.
• Infoaeo.organization-missingThe home page has no Organization (or Person) structured data.
• Infoaeo.entity-same-as-missingOrganization or Person structured data has no sameAs links to the entity's profiles elsewhere.
• Erroraeo.robots-blocks-search-crawlerrobots.txt blocks a search crawler whose index AI answers draw on: Googlebot (AI Overviews, AI Mode), Bingbot (Copilot) or Applebot (Siri, Spotlight).
• Warningaeo.robots-blocks-answer-botrobots.txt blocks a bot that fetches pages for AI answers or an AI search index (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, DuckAssistBot, MistralAI-User, Amzn-SearchBot, Amzn-User, meta-webindexer, meta-externalfetcher, MistralAI-Index, YandexAdditional).
• Infoaeo.robots-blocks-training-botrobots.txt blocks a crawler that collects text to train AI models (GPTBot, ClaudeBot, Applebot-Extended, CCBot, meta-externalagent, Amazonbot, MistralAI-Training).
• Warningaeo.content-signal-blocks-answersA robots.txt Content-Signal line sets ai-input=no or search=no, asking answer engines not to use the pages in AI answers or search.