GitHub Add to your assistant
Menu

site.robots-blocks-resources

WarningSEO auditcrawl

robots.txt keeps Googlebot from scripts or stylesheets the pages load, so Google renders the pages without them.

What to change

Allow the CSS and JavaScript files pages need (Allow: /assets/, or remove the Disallow line for that folder). Keep blocking only what pages do not need to render.

The documentation on developers.google.com

A finding of this rule on a page carries the exact edits for that page: the markup it quotes, and what to write instead.

When it runs

Runs on a crawl or a build directory. A warning is a problem that costs a page ranking, clicks or citations. Each finding takes 5 points off the page's score.

The severity here is the rule's usual one: a finding can be raised or lowered when the page gives a reason.

Run it, or skip it

Check a page for this rule alone, after a fix for instance:

npx @rivi-software/agentic-seo-check crawl https://example.com --rule site.robots-blocks-resources

Or from an assistant, with the MCP tool seo_crawl_site:

{
  "url": "https://example.com",
  "rules": [
    "site.robots-blocks-resources"
  ]
}

To leave it out of an audit, pass --disable site.robots-blocks-resources on the command, or disableRules: ["site.robots-blocks-resources"] to a tool.

All 49 rules