robots.txt is reached through more than five redirects

Why it matters

A long redirect chain adds requests before a crawler can read robots.txt. Google documents support for at least five hops and treats a chain it stops following as a missing file. This check records the chain observed by the audit; it does not establish where Google would stop.

What to do

1. Look at the finding: it lists each redirect response in order, with its address, status and Location value.
2. Make /robots.txt answer 200 at your site address, or point it straight at its final address with a single redirect.
3. Check redirect rules that apply to every address, such as http to https, a www change and a trailing-slash rule; together they can add several steps before the file is served.

What we read

At the start of the audit we request /robots.txt at your site address over HTTPS and follow redirects, recording the address, status and Location value of every redirect response we receive. More than five such responses are reported. The count uses only responses that were actually received, including a chain that never reaches a file.

A followed redirect must stay on the exact HTTPS site origin or its twin with or without www, at the same effective port. The terminal response is not an extra redirect. A complete terminal rules file also supplies the count of nonempty Disallow rules.

Example & evidence

Illustrative example — one audit, not a live result.

An audit of bakery.example requests the robots.txt file and receives six redirects before the file:

GET https://bakery.example/robots.txt        301 -> /robots.txt/
GET https://bakery.example/robots.txt/       301 -> https://www.bakery.example/robots.txt/
GET https://www.bakery.example/robots.txt/   301 -> /robots.txt
GET https://www.bakery.example/robots.txt    302 -> /files/robots.txt
GET https://www.bakery.example/files/robots.txt   301 -> /static/robots.txt
GET https://www.bakery.example/static/robots.txt  301 -> /static/robots-v2.txt
GET https://www.bakery.example/static/robots-v2.txt  200 text/plain, 3 Disallow rules

Six redirect responses are more than five, so the site is reported. A possible correction: https://bakery.example/robots.txt answers 200 directly, and nothing is reported.

Confirm & check again

Check for yourself: Request /robots.txt on your address with a tool that shows every redirect, such as the Network tab of your browser's developer tools, and count the redirect responses before the file arrives.

Check again: Full audit — Run a new audit of the same site. A complete robots.txt file or a 404 or 410 reached through five received redirects or fewer can fix this finding on the same site identity. An incomplete walk, unknown terminal evidence or failed audit admission never proves a fix or replaces the last eligible report.

Limits & sources

Only more than five received redirects triggers this finding. An off-host redirect or an HTML landing does not itself trigger it. Redirects outside the admitted HTTPS origin or its www twin are refused before the target is requested.

With five or fewer redirects, HTML, an incomplete file, a 429/5xx/401/403 response or an uncompleted walk stays unknown. More than five received redirects is positive redirect-count evidence even when the terminal is unqualified; audit admission still decides whether a report can be published.

The check uses the existing bootstrap acquisition and its recorded attempts; it adds no probe. JavaScript, frames and meta refresh are not followed. One audit vantage does not establish behavior for other clients or at other times.

Confidence: High Every counted redirect is an HTTP response the audit received and recorded with its status and Location value. The five-hop threshold belongs to this product; it does not establish a Google maximum.

Sources

• Google, How Google interprets the robots.txt specification — Documents at least five supported robots.txt redirect hops and how Google handles a chain it stops following; it does not specify five as a hard maximum. JavaScript, frame and meta refresh redirects are not followed for this file. Checked 14 October 2026.

Editorial review: 2 November 2026