RankEcho

Crawling and indexing

What are 5xx errors and why do crawlers care?

A 5xx status code means the server failed: 500 internal error, 502 bad gateway, 503 unavailable. Crawlers read repeated 5xx errors as a struggling site and slow down, and pages that keep failing drop out of the index. A 503 with Retry-After is the right answer for planned downtime.

How to read it

One failure is forgiven. A run of them makes Google crawl less, and a robots.txt that returns 5xx can stop crawling of the whole site until it loads again.

AI fetchers have no patience at all: a page that fails when a user asks is simply missing from that answer.

Example from rankecho.io

0 server errors to crawlers in 30 days

Across every crawler we track, 30,984 verified requests reached rankecho.io in 30 days, and none got a 5xx status.

That is the goal for any site: crawlers that never meet a failing server keep coming at full speed.

Source: rankecho.io's Cloudflare logs, verified requests only, 17 October to 15 November 2026.

Questions people ask

What are 5xx errors and why do crawlers care?

A 5xx status code means the server failed: 500 internal error, 502 bad gateway, 503 unavailable. Crawlers read repeated 5xx errors as a struggling site and slow down, and pages that keep failing drop out of the index. A 503 with Retry-After is the right answer for planned downtime.

How quickly do 5xx errors affect rankings?

A few hours of errors rarely matter. Days of them can drop pages, which usually come back once the site is healthy.

What should a site return during maintenance?

503 Service Unavailable with a Retry-After header, never a 200 maintenance page.

Related terms

Read more on rankecho.io

See it on your own site

RankEcho reads your Search Console data and asks AI assistants whether they cite you, then shows each of these numbers for your pages. The audit is free.

Definition and example checked 2026-11-16.