Crawl budget8 rules onwhat may waste crawl time on your site.
Server strain, page size, revalidation and filter addresses that multiply.
Server load
- Many site-origin requests returned 5xx or 429 during the crawlThe share of site-origin crawl attempts returning 5xx or 429 meets the configured threshold; correlated bursts may need review.In the report: Elevated 5xx/429 rate observed during this crawl
- Site-origin responses are slow during the crawlThe p75 transport response time of attributable site-origin replies in the audit sample exceeds the configured threshold.In the report: Slow host responses risk Googlebot crawl capacity
Page size
Revalidation
Filters and traps
- A chain of generated addresses repeats the same main textFour original links lead through five usable responses with the same main-text hash under one supported address pattern.In the report: A chain of generated addresses repeats the same main text
- An empty filter result returns HTTP 200A proved filter address returns 200 with an explicit zero-result indicator in its captured main region.In the report: Zero-result filter pages return 200 instead of 404
- Filter addresses use inconsistent separators or orderingA proved filter pattern exposes literal nonstandard field separators or duplicate query orderings with the same recorded main content.In the report: Facet parameters use non-standard separators or inconsistent ordering, multiplying duplicate URLs
- Many filter addresses share the same main text in our sampleA large set of discovered filter addresses has few distinct main texts in the recorded sample.In the report: Many filter addresses share the same main text in our sample