A chain of generated addresses repeats the same main text

crawler_trap_url_pattern · Issue

Short answer

We recorded a sequence of five addresses connected by four actual links. Their main texts are identical and their addresses follow one supported generator pattern. Review and stop the unwanted generator. The finite sequence does not prove that the site has infinitely many addresses.

Severity: Medium. Repeated generated addresses deserve attention, but check whether the views are intentionally useful before removing them.

Why it matters

Google recommends reducing unnecessary duplicate addresses so crawling is spent on useful pages.

What to do

  1. Follow the recorded links and compare the five responses. Identify the template or control generating the next address.
  2. Correct the generator: resolve relative links to the intended address, cap pagination or calendar ranges, or stop exposing unnecessary session identifiers. Restrict unwanted addresses in robots.txt when they should not be read.
  3. Run a new full audit. Changing only the displayed representative, canonical or noindex tag does not demonstrate that the original sequence is gone.

When to leave it as it is: Leave the finding after review if these are useful views and their identical static text is intentional. Confirm the actual rendered content and links.

What we read

We reuse original captured links and usable main texts with a separately captured robots.txt and sitemap sources. Supported generators include literally self-appending paths and locally declared pagination, calendar or session controls. Parameter names or address depth alone do not establish a generator.

A finding needs four actual links and five usable responses with identical main-text hashes. Each intermediate outgoing link must belong to the same captured response reached by the preceding link. A real redirected address can start the sequence; another alias cannot supply a missing intermediate link.

Filter patterns belong to the separate filter-address rule. Discovery depth counts describe this audit’s observations; they do not prove an infinite address space.

Example & evidence

Illustrative pagination sequence, not a live audit.

/catalog?page=1 → /catalog?page=2 → /catalog?page=3
                → /catalog?page=4 → /catalog?page=5

Each arrow is an actual recorded next-page link under a local pagination declaration. All five usable responses have the same main-text hash. This is evidence of a repeated finite sequence, not proof of infinity.

Correct the generator and run a full audit that reads the retained original sources. If one endpoint times out, that missing response does not prove the chain was fixed.

Confirm & check again

Check for yourself: Follow the original sequence, inspect the next-link targets and compare the main content. Check whether another original link keeps an equivalent sequence alive.

Check again: Full audit — Run a new full audit. It independently compares retained original sources, edges, usable main texts, robots permission and sitemap dependencies. A confirmed removed or changed generator, unequal usable endpoint texts or explicit denial can break an edge. A surviving alternative four-link sequence prevents a fix. Missing sources or coverage do not close the finding.

Limits & sources

The check is a static, high false-positive-risk heuristic. Identical main text can omit meaningful images, data or JavaScript content. Five responses do not establish infinity, spam intent, a search penalty or measured Google crawling. Reaching our audit limit is coverage information, not a site finding.

Confidence: Medium The linked responses and equal hashes are recorded; the usefulness of the repeated views still needs judgement.

Sources

  • Google, Optimize your crawl budget — Recommends managing unnecessary duplicate addresses. This finite five-response predicate is our heuristic, not a Google test or threshold. Checked 8 October 2026.

Editorial review: 2 November 2026