This guide explains what the new __usage_ai-training rule actually says, what it does not do, when the companies say they will start reading it, and how to add it to WordPress safely. Facts below were checked against the original announcements and documentation on 10 November 2026. Dates and support are still moving, so treat this as a snapshot rather than a final standard.

What Was Announced on 30 October 2026

Three announcements landed on the same day, all pointing at one Internet-Draft:

The Verge also reported that Anthropic, where the draft’s author works, intends to adopt it. I have not found a separate Anthropic announcement, so I would not rely on a date for that yet.

Timeline of the shared AI training signal from the 30 October 2026 announcements to Bing support on 1 February 2027 and the 1 March 2027 NoArchive deadline
Nothing reads the shared rule today. Google says early 2027; Microsoft gives 1 February 2027.

How the Robots.txt AI Training Opt-Out Rule Works

The draft adds no new directives and no new grammar. You write a normal group, but the User-agent line names a purpose instead of a crawler:

User-agent: __usage_ai-training
Disallow: /
Allow: /press/

Under the draft, a disallowed path carries a preference against AI training and an allowed path carries a preference in favour. That example asks participating providers not to train on anything except the /press/ folder. Path matching and longest-match precedence follow the existing robots.txt standard, RFC 9309, so the rules behave the way you already expect.

Three details matter more than the headline:

Annotated WordPress robots.txt showing a User-agent star group for normal crawling and a separate __usage_ai-training group with Disallow slash and an Allow exception
Keep the new token in its own group so your search rules stay exactly as they were.

What the Shared Rule Does Not Change

The biggest risk is assuming this rule does more than it does. It is a preference about training, read only by providers that choose to honour it. It is not a privacy wall and it is not an AI search switch.

If your real worry is AI Overviews rather than model training, that is a different set of controls. My guide to AI Overviews opt-out options covers what you can and cannot change there.

Three-column summary of what the shared AI training rule does, what it does not do, and which bot and grounding controls remain separate
Training preference, crawl access and AI answer visibility are three different decisions.

Decide Training and AI Visibility Separately

Before you touch the file, decide what you actually want. For most small business sites I work on, the answer is “stay visible everywhere people search, and think carefully about training”. You might reasonably decide either way on training. Being cited in AI answers can send visitors; training gives you nothing direct.

Work through the choices in this order:

  1. Search first. Never block Googlebot or Bingbot as a way to stop AI training. You would lose organic search, and the shared token makes that blunt approach unnecessary.
  2. Training next. Opt out with the shared token, opt out for some folders only, or leave it unset.
  3. AI answers last. Decide on Google-Extended for Gemini grounding, OAI-SearchBot for ChatGPT search, and Microsoft’s page-level grounding preference. OpenAI’s documentation says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers.

If being cited is part of your plan, read how to get cited in AI Overviews and AI Mode before you opt out of anything.

Four-step decision path: keep search crawlers open, choose a training preference, choose AI answer visibility, then write and test robots.txt
Making the decisions in this order stops a training opt-out from quietly costing you search or citations.

How to Add It in WordPress

WordPress can serve robots.txt three ways, and you need to know which one your site uses before editing anything:

Open https://yoursite.com/robots.txt in a private browser window and compare it with what your plugin shows. If they differ, a physical file or a host rule is winning. Then add the new group at the end, after your existing rules:

# Shared AI training preference (IETF draft, Oct 2026)
User-agent: __usage_ai-training
Disallow: /

On a custom build where you prefer code, a small filter in a site plugin does the same job for the virtual file:

add_filter( 'robots_txt', function ( $output, $public ) {
    if ( $public ) {
        $output .= "\nUser-agent: __usage_ai-training\nDisallow: /\n";
    }
    return $output;
}, 20, 2 );

Do not stack the token under User-agent: *. Consecutive User-agent lines share one group in robots.txt. If you put __usage_ai-training directly below User-agent: * and then write Disallow: /, you have also told every search crawler to leave. Give the token its own block with a blank line before it.

Keep Your Existing Bot Rules for Now

None of the three companies says the shared token replaces its own controls. Google says company-specific controls “will continue to function alongside this new joint signal”. OpenAI says its existing crawler controls continue to apply. Microsoft is replacing its older NoArchive and NoCache handling for training and grounding, which is a separate change.

So for the next few months:

Bing also notes that publishing a preference is not the same as activating it. A page has to be recrawled before the setting is discovered.

Test the File After Every Change

Robots.txt mistakes are quiet. Nothing breaks visibly; traffic just drifts down weeks later. After you publish the new group:

Eight-point WordPress robots.txt AI checklist covering the live file, who writes it, a separate token group, named bot rules, search visibility and Search Console testing
Run this checklist after any SEO plugin, hosting or theme change, not just today.

Robots.txt is one line in a larger technical picture. My WordPress technical SEO checklist covers the rest, and the llms.txt guide explains why that separate file is not an access control either.

Frequently Asked Questions

What is __usage_ai-training in robots.txt?

It is a policy token defined in an October 2026 IETF Internet-Draft. A robots.txt group with that token as its User-agent states your preference about AI model training for the paths it covers. Disallow means you prefer no training; Allow means you are in favour.

Will adding the shared AI training rule hurt my Google rankings?

Not if you add it as its own group. Google says Google-Extended, which the token maps to, does not affect inclusion in Google Search and is not a ranking signal. Problems only start if you merge it with User-agent: * or block Googlebot itself.

Does the shared rule work right now?

No provider has said it reads the rule yet. As of 10 November 2026, Google says early 2027, Microsoft says from 1 February 2027, and OpenAI says in the coming months. Publishing it now is harmless and means it is in place when support starts.

Should I remove my GPTBot and Google-Extended rules?

Not yet. Google and OpenAI both say their existing controls continue to work alongside the shared signal, and they are the rules being read today. Review them again once each provider confirms its implementation.

Does opting out of AI training remove my site from AI answers?

Not by itself. Training and answer use are separate choices. Google-Extended also covers Gemini grounding, OpenAI uses OAI-SearchBot for ChatGPT search, and Microsoft has a separate page-level grounding preference.

Final Thoughts

The shared robots.txt AI training opt-out is a sensible simplification: one rule instead of a list that never stays current. It is also a draft, not a finished standard, and nothing reads it yet. Add it as a separate group, keep your existing bot rules, protect your search crawlers, and recheck the file once Google, Microsoft and OpenAI confirm their implementations in 2027.

For the original announcement and what Google said it will support, see my news report: Google plans to honour a shared AI training opt-out in robots.txt.

If you would like someone to review your robots.txt, AI crawler settings and wider technical SEO in one pass, my SEO improvement service covers exactly that kind of audit.

Official References

Sources checked on 10 November 2026: