In response to web publishers’ feedback, Google has introduced a novel feature, Google-Extended enables them to exclude from AI development while maintaining their prominence in Google Search.
Harmonizing AI Development with Search Presence
Google-Extended guarantees that websites stay searchable and indexed by bots such as Googlebot, yet their information won’t feed into Google’s AI training. This effort chiefly focuses on the growth of Google’s AI chatbot, Bard, which, as acknowledged in July, draws from online data.
How Does It Work?
Available through robots.txt, which control web crawler access, Google-Extended provides publishers an option. They now have the choice to determine whether their material aids in the enhancement of “Bard and Vertex AI generative APIs.” Furthermore, it provides an avenue for publishers to oversee content availability.
User-agent: Google-Extended
Disallow: /The Broader Context
As AI’s role in the digital realm grows, the debate over data usage intensifies. Major platforms like The New York Times, CNN and Reuters have already restricted the web crawler used by OpenAI for ChatGPT. The challenge? Blocking Google’s crawlers entirely means vanishing from search results. This challenge led entities like The New York Times to legally bind Google by revising their terms of service, barring the utilization of their material for AI development.
Google acknowledges the expanding applications of AI and commits to further enhancing digital selection options for web publishers. As Google hints at more developments in the pipeline, the digital community keenly awaits the next chapter in this evolving narrative.
