Cloudflare gives AI crawlers an ultimatum: split search from training or get blocked
Cloudflare just gave AI companies until September 15, 2026, to decouple crawlers used for traditional search from those powering AI training and agents. Miss the deadline, and mixed-use bots will be blocked by default across ad-supported sites. The move puts an end to tech giants scraping free training data under the guise of web indexation, while expanding a model where publishers get paid when content actually creates value.