The Dirty Secret of AI Search

As Bing, Google, and Baidu announce major overhauls of their search engines to incorporate AI-powered capabilities, the environment might be worse off for it.

According to a report on Wired, integrating large language models such as ChatGPT into search engine could result in as much as a fivefold increase in computing power – dramatically increasing carbon emissions.

“Training these models takes a huge amount of computational power,” said Carlos Gómez-Rodríguez, a computer scientist at the University of Coruña in Spain. “It’s not that bad, but then you have to take into account [the fact that] not only do you have to train it, but you have to execute it and serve millions of users.”

At the heart of the issue is the dramatic increase in scale when integrated into search engines such as Bing, which handles half a billion searches a day, compared to the millions of users that utilise ChatGPT daily.

The Wired report cited Martin Bouchard, cofounder of Canadian data center company QScale who estimates that the incorporation of generative AI to search engines will require a minimum of “four or five times” more computing per search.

This makes sense when one considers how ChatGPT was trained from a curtailed cache of data restricted to late 2021 and earlier, while search engine versions would presumably have to be retrained frequently to incorporate the latest information that appear on the Internet.

The sharp increase in compute needed to deliver AI-powered search is also a challenge.

This explains the gradual roll out of the refreshed Bing service. For all of Microsoft’s global infrastructure with hundreds of data centers, the rollout of its new AI-powered chat service is being done gradually around the world. Moreover, the service declines further conversation beyond a certain point – presumably to preserve its capacity.

To be clear, many of the technology giants have committed to renewables. For instance, Microsoft has committed to purchasing renewable energy with a goal of achieving 100% coverage of electricity consumption with renewable energy by 2025.

On its part, Google – which has already achieved 100% renewable energy – claims that it can reduce the carbon footprint of a such system by as much as 1,000 times by combining efficient ML models, processors, and data centers with clean energy sources.

The spokesperson who responded to the query from Wired cited research published by Google researchers together with a researcher from Berkeley in 2021, which shows that it is a problem that Google is well-aware of. You can read the paper titled “carbon emissions and large neural network training” here.

Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].​

Image credit: iStockphoto/Artur Nichiporenko