The US government is considering its options Landmark “monopoly” ruling against Google last week, online publications face an increasingly bleak future. (And this time, it’s not just about ad revenue plummeting.) Bloomberg reports Their choice is now to allow Google to use the published content to produce it inline AI-generated search “answers” or losing the company’s search engine visibility.
The crux of the problem lies with Googlebot, which crawls and indexes the live web to get the results you see when you enter search terms. If publishers prevent Google from using their content for the AI-engineered answers you now see scattered at the top of many search results, they also lose the privilege of including their web pages in standard web results.
The Catch-22 publications have prompted rival search engines and AI startups to pin their hopes on the Justice Department. Tuesday, The New York Times informed The DOJ is considering asking a federal judge to break up parts of the company (unbundling divisions like Chrome or Android). Other options reportedly being considered include forcing Google to share search data with competitors or opting out of default search engine deals. He signed an 18 billion dollar deal with Apple.
Google uses a separate scanner for this Gemini (formerly Bard) chatbot. But its core browser includes both AI views and standard searches, leaving web publishers with little (if any) leverage. If you allow Google to scrape your content for AI browsing responses, readers may think it’s dead (which means zero revenue from potential readers) without bothering to visit your site. But if you block Googlebot, you lose search visibility, which likely means a significant loss of short-term revenue and a huge loss of long-term competitiveness.
iFixit CEO Kyle Wiens said Bloomberg“I can block ClaudeBot [Anthropic’s crawler for its Claude chatbot] from indexing us without harming our business. But if I block Googlebot, we lose traffic and customers.”
Another problem with combining the two is that it gives Google an immeasurable advantage over smaller AI startups. The company receives tons of free training information from publishers eager to appear in search. In contrast, AI companies are forced to pay publishers to access their data — and even then, it won’t add to Google’s (mostly) free parent product.
In this regard, it is not surprising to read this BloombergGoogle rejects publishers trying to negotiate content deals. (Reddit was the only exception.) Why spend money on content deals when most publishers can get all the training they want in exchange for the search results they need to survive?
“There are a lot of tech companies that are paying for content now, they’re paying for access to it because they need it to be able to compete seriously in any way,” said Alex Rosenberg, CEO of AI startup Tako Inc. , he said Bloomberg. “And for Google, they don’t really have to.”
This is due to the leverage Google has over desperate publishers. On top of the industry’s current financial woes (online ad revenue has fallen off a cliff over the past eight years), Advertising Week informed In March, Google’s AI-generated search answers could cause a 20-60 percent drop in organic search traffic.
The ball is now in the Justice Department’s court to figure out where Google, and to some extent the entire internet, goes from here. Bloombergis the full story worth reading.
