OpenAI,Google,Microsoft，和Anthropic表示，他们将采用一种新的标准，出版商可以选择退出被AI刮去，如之前搜索引擎圆桌会议所报道的那样。 有了这种控制，出版商可以将其AI培训偏好适用于多个AI公司，而不是单独设定。
AI中最大的名字支持一种新的方法选择退出刮取
新控制将允许出版商在多个爬行者身上应用他们的AI培训偏好。
共有的“政策标志”信号由Anthropic标准技术领导Kevin Kelley提出，作为因特网工程工作队的一部分。 它将生活在一个网站的机器人中。 txt文件，它已经长篇指示了哪些网络爬行者可以和不能刮去一个站点，无论是搜索索引、存档还是人工智能培训。 目前，网站必须向每个AI公司分别表示他们的AI培训偏好。
出版商在不断涌入AI爬行者捣毁其网站并将其内容输入AI模型时，一些公司和组织试图打击未经授权的刮刮行为，采用"真正简单的许可证(RSL)"标准，让出版商能够向访问其网站的爬行者收取费用，而Cloudflare默认封锁了AI爬行者。
OpenAI,Google,Microsoft，和Anthropic表示，如果遇到共享的会令牌，他们会限制他们的AI爬行者访问一个站点。 然而，Microsoft将提供更多的颗粒控制，使出版商能够选择退出AI培训或AI地平线，这一过程将AI模型指向一个经过核实的信息来源来减少幻觉。 Microsoft 指出，网站无论喜好如何，仍将出现在Bing的搜索结果中。
虽然Anthropic已经采纳了信号，但Google和Microsoft计划明年支持。 OpenAI 称将在“未来几个月”内实施。
本文由 AI宝库自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually.
The biggest names in AI are backing a new way to opt out of scraping
The new control will allow publishers to apply their AI training preferences across multiple crawlers.
The shared “policy token” signal is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live within a website’s robots.txt file, which has long signaled which web crawlers can and can’t scrape a site — whether for search indexing, archiving, or AI training. Currently, sites have to signal their AI training preferences separately to each AI company.
The announcement comes as publishers grapple with the continuing influx of AI crawlers scraping their sites and feeding their content into AI models. Some companies and organizations have attempted to combat unauthorized scraping, with the Really Simple Licensing (RSL) standard giving publishers the ability to charge crawlers that visit their sites, and Cloudflare blocking AI crawlers by default.
OpenAI, Google, Microsoft, and Anthropic say they will restrict their AI crawlers from accessing a site if they encounter the shared token. Microsoft, however, will offer more granular controls that will give publishers the ability to opt out of AI training or AI grounding, the process of pointing an AI model to a verified source of information to reduce hallucinations. Microsoft notes that websites will still appear in Bing’s search results regardless of their preferences.
While Anthropic has already adopted the signal, Google and Microsoft plan to support it next year. OpenAI says it will implement it in the “coming months.”
Auto-fetched and translated by AI Gem Box. All rights belong to the original author.