Many site-origin requests returned 5xx or 429 during the crawl

Why it matters

Google's page on HTTP status codes says 5xx and 429 errors make its crawlers temporarily slow down, counts 429 as a server error, and says URLs already indexed are kept at first but eventually dropped.

What to do

1. Look at the finding: it shows aggregate error counts, the attempt denominator, status totals, retained Retry-After values and a summary of request timing correlation.
2. Check your server's error logs for the time of the audit: memory or database limits, a failing plugin or a firewall rate rule are common causes.
3. If a firewall or CDN answers 429, make sure it does not turn search crawlers away at a normal pace.

When to leave it as it is: if the finding is kept for review because our faster requests set off your rate limit, and your logs show search crawlers are not refused, you can leave it.

What we read

No extra requests are made: we use the audit's own requests for your pages, retries included, and leave out requests for robots.txt, sitemap files and other checks. The crawler retries once after a 5xx, a 429 or a dropped connection, and each attempt counts. Requests that got no answer at all, such as timeouts, count in the total but not as errors.

The site is reported when both the configured share of same-origin attempts, 10% by default, and the configured number of 5xx or 429 answers, three by default, are reached, judged over at least 10 responses. These are audit settings, recorded with each audit. A 403 or another 4xx is not counted as an error.

With fewer answers than the minimum, the check can only clear the site, never report it, and only after an audit of the whole site that fetched every page it found, without leaving pages out because of the page limit and with no request left unanswered: the site is clear when fewer 5xx or 429 answers than the configured number came back; otherwise the check cannot conclude. An audit of selected pages cannot conclude with fewer answers. In every case the check cannot conclude when the timing of the requests was not fully recorded, or when a single crawled address redirected, even only to add a trailing slash or to move from http:// to https://, or when an answer came from another host: then not every request can be put down to your server.

Example & evidence

Illustrative example — one audit's answers, not a live audit.

An audit of https://bakery.example/ made 60 requests to the site:

200 OK: 51
503 Service Unavailable: 7
429 Too Many Requests: 2

Nine of 60 attempts, 15%, returned 5xx or 429, meeting both example settings (10% and three). The finding shows aggregate status counts and timing summaries, not a timestamp for each error.

If only two answers had been errors, there would be no finding: fewer than three are not reported.

Confirm & check again

Check for yourself: look in your server's error log or hosting dashboard for 5xx and 429 responses during the audit window.

Check again: Full audit — run a new full audit. With the minimum response population and complete telemetry coverage, the finding closes when either the error share or the error count falls below its recorded threshold. On a small site, an audit of the whole site that fetches every page it finds, with fewer answers than the minimum and no request left unanswered, also closes it when fewer 5xx or 429 answers than the configured number come back. A crawl with a coverage gap, such as an address that redirected, cannot confirm it; a finding that moves between issue and review remains the same finding.

Limits & sources

The count covers our crawler's requests in one audit, at its pace, from one place: it is not Googlebot's error rate, and it does not prove that pages were dropped from the index. When the errors follow our pace, the check cannot rule out a limit aimed at our crawler alone.

Confidence: Medium because one audit is a short sample of how the server answers.

Sources

• Google, How HTTP status codes affect Google's crawlers — says 5xx and 429 errors make Google's crawlers temporarily slow down, treats 429 as a server error, and says indexed URLs are preserved but eventually dropped. Checked 28 September 2026.

Editorial review: 29 October 2026

Related

• A page fails with a server error — one page keeps failing with a server error.
• Site-origin responses are slow during the crawl — the server answered slowly rather than with errors.