Real numbers. No hype. Receipts.
Here is a claim moving fast right now: “Starting September 15, Cloudflare blocks AI bots by default on new websites — so your new site will be invisible to AI.” If you just registered a domain, or you are about to, that sounds like a wall dropping in front of you.
We pulled the receipt. It is Cloudflare’s own announcement, not a summary of a summary. The real rule is narrower — and in most cases far less scary — than the headline.
What Cloudflare actually said
On September 15, 2026, Cloudflare changes the default AI-crawler settings for new domains onboarding to Cloudflare. It sorts AI crawlers into three jobs:
- Search — indexes your content so an assistant can answer questions about it and, ideally, send people back to you.
- Agent — acts in real time on a person’s behalf (a chatbot or browser agent fetching your page live).
- Training — takes your content to train or fine-tune a model.
For a new domain, the new default is: Training and Agent are blocked — but only on pages that display ads. Search stays allowed. Existing domains are not flipped automatically; Cloudflare lets current customers set their own choice in Security settings any time before September 15.
The part the headline dropped
Two words change the whole story: “display ads.” The default block on Training and Agent applies to pages that show ads. If your site does not run ads, there is nothing for that default to switch off. And the one category that actually decides whether an assistant can find and cite you — Search — is allowed by default either way.
So the honest version is: a new, ad-free site is barely touched by this change, and the crawler that gets you seen stays open. “Invisible to AI by default” is not what the document says.
Who should actually care
This is a real, pointed change for one group: ad-monetized publishers on new domains. If your pages carry ads, Training and Agent crawlers get blocked for you out of the gate — which is arguably the point, since that is Cloudflare handing publishers leverage to charge AI companies for their content. There is also one edge to know: multi-purpose crawlers that do both Search and Training (Googlebot is the classic example) get caught by the Training block. Blocking Training can mean blocking a crawler you actually wanted for Search. That is the trade-off worth checking, not a reason to panic.
What to do — two minutes in Security settings
If you own or are about to launch a new domain on Cloudflare, do not guess. Open Security → Bots (AI crawler controls) and confirm, in writing, which of the three categories — Search, Agent, Training — you are allowing and blocking, and on which pages. Make it a decision, not a default you inherited. If you run no ads and you want to be found and cited, leaving Search allowed is the setting that matters. If you monetize with ads, decide deliberately whether the Training/Agent block is what you want, and watch the multi-purpose-crawler edge.
That is the whole receipt. The deadline is real, the date is real, and the direction — sites getting the controls to decide who crawls them — is worth positioning for. But “new domains go dark to AI” is a scare, not a fact. Check the box that applies to your site and move on.
Sources (primary): Cloudflare Blog, “Your site, your rules: new AI traffic options for all customers” (July 1, 2026); Cloudflare Developer Changelog, “New options to manage AI traffic” (July 1, 2026). Effective date for new-domain defaults: September 15, 2026.
Curates public information from primary sources for education. Not investment, financial, legal, or tax advice. — Trunkline · Carter Enterprise LLC
Leave a Reply