Google-Extended is disallowed: review its usage scope
In the reportGoogle-Extended is disallowed; review its Gemini usage scope
google_extended_lever_scope_mismatch · Review item
Google documents Google-Extended as a control of Gemini training and grounding usage. It is not a separate crawler and does not determine Google Search inclusion or ranking. A sitewide restriction may be exactly the policy you want.
Severity: Notice. Informational advice: an intentional usage restriction does not mean your configuration is wrong.
Why it matters
Usage permission and crawler access are different controls. This note helps you assess the documented Google-Extended scope without treating it as a general switch for Google Search.
What to do
- Compare the exact named Google-Extended group and captured Disallow lines with your intended Gemini usage policy.
- Leave an intentional restriction in place. If you want to remove it, use an explicit full Allow or establish complete unrestricted coverage; moving the deny to wildcard is not a fix.
When to leave it as it is: Leave the restriction if it matches the Google-Extended usage policy you want.
What we read
One original HTTP 200 robots.txt from the registered site origin, without a redirect, in UTF-8 and within the 500 KiB policy window. Its complete retained bytes and declaration index are resolved against the frozen admitted bot reference.
The notice needs a proven all-path restriction through an exact named Google-Extended group. A material wildcard-only deny or a named partial-path restriction stays unknown and cannot fix an earlier finding.
Example & evidence
Illustrative named usage control, not proof of training or a search result change.
User-agent: Google-Extended
Disallow: /If this matches your Gemini usage policy, no usage-policy change is required.
Confirm & check again
Check for yourself: Open the original robots.txt and compare the named agent and wildcard groups, including every Allow exception.
Check again: Full audit — Run a new complete audit of the same site. Fully checked unrestricted Google-Extended usage permission can fix the same origin finding. A partial exception, moved deny, unavailable or redirected robots.txt, incomplete retained bytes, or missing or stale bot reference cannot prove a fix. If robots policy denies the audit itself, the previous report remains in place.
Limits & sources
The check does not inspect Googlebot, page snippet controls or actual Google requests.
This rule makes no AI Overviews or AI Mode conclusion; their separate controls are outside the admitted robots-only evidence.
A wildcard deny is not attributed to a deliberate separate Google-Extended choice.
A file beyond the 500 KiB policy window, an incomplete declaration index, non-UTF-8 encoding or an HTML response cannot establish a clean result. The audit uses its own user agent; it does not send a Google-Extended request.
Confidence: High The captured declarations and reviewed publisher metadata establish the captured usage-control declaration, not actual fetching or use.
Sources
- Google, Google common crawlers — Documents Google-Extended usage-control scope and robots group controls; the token does not affect Google Search inclusion or ranking. Checked 9 September 2026.
- Google, Google robots.txt documentation — Documents robots group selection and Allow/Disallow permission controls. Checked 9 September 2026.
Editorial review: 2 November 2026