AI Giants Adopt New Standard for Data Scraping Opt-Out

**AI Giants Adopt New Standard for Data Scraping Opt-Out** In a significant move towards greater control over content usage, major artificial intelligence companies including OpenAI, Google, Microsoft, and Anthropic have announced their commitment to adopting a new standard that will empower publishers to easily opt out of having their content scraped by AI models. This development, initially reported by Search Engine Roundtable and further detailed by The Verge AI, marks a collaborative effort to address growing concerns around data sourcing for AI training. The proposed standard introduces a shared "policy token" signal, an initiative spearheaded by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). This token will be integrated into a website's `robots.txt` file, a long-standing mechanism used by webmasters to signal which web crawlers are permitted or prohibited from accessing and indexing their site's content. The primary benefit of this new approach is its ability to streamline the process for publishers. Instead of having to set individual AI training preferences with each AI company, publishers will now be able to apply their desired AI scraping policies across multiple AI companies simultaneously. This unified control mechanism is expected to provide a more efficient and transparent way for content creators to manage how their intellectual property is utilized in the development and training of advanced AI systems, whether for search indexing, archiving, or other AI-driven applications.