aeo.crawl-stopped

InfoAEO audit

The hosted server stopped requesting the site before the crawl was done: the site refused its requests (a challenge, HTTP 429), robots.txt asked for a Crawl-delay longer than a hosted crawl waits, or the server reached its request limit for the site. The audit covers the pages it read.

What to change

The message says which. The hosted server's own limit for a site lifts within a minute: crawl again then. A refusal or a Crawl-delay is the site's choice, which the hosted server keeps.

A finding of this rule on a page carries the exact edits for that page: the markup it quotes, and what to write instead.

When it runs

Runs on every AEO audit: of a page, a crawl, a build directory or HTML. Info is a best practice: worth doing, rarely urgent, and --no-info leaves it out. Each finding takes 1 point off the page's score.

The severity here is the rule's usual one: a finding can be raised or lowered when the page gives a reason.

Run it, or skip it

Check a page for this rule alone, after a fix for instance:

npx @rivi-software/agentic-seo-check aeo https://example.com --rule aeo.crawl-stopped

Or from an assistant, with the MCP tool seo_aeo_audit:

{
  "url": "https://example.com",
  "rules": [
    "aeo.crawl-stopped"
  ]
}

To leave it out of an audit, pass --disable aeo.crawl-stopped on the command, or disableRules: ["aeo.crawl-stopped"] to a tool.

All 32 rules