Google joins push for a single AI training switch in robots.txt
Google just agreed to adopt the IETF AIPREF standard alongside OpenAI, Anthropic, and Bing. Instead of blocking every AI crawler one by one, website owners will soon be able to use a shared policy token in robots.txt to opt out of training across all participating providers. Google plans to roll out support in early 2027.