“It’s not that bad, but then you have to take into account [the fact that] not only do you have to train it, but you have to execute it and serve millions of users,” Gómez-Rodríguez says.
There’s also a big difference between utilizing ChatGPT—which investment bank UBS estimates has 13 million users a day—as a standalone product, and integrating it into Bing, which handles half a billion searches every day.
Martin Bouchard, cofounder of Canadian data center company QScale, believes that, based on his reading of Microsoft and Google’s plans for search, adding generative AI to the process will require “at least four or five times more computing per search” at a minimum. He points out that ChatGPT currently stops its understanding of the world in late 2021, as part of an attempt to cut down on the computing requirements.
In order to meet the requirements of search engine users, that will have to change. “If they’re going to retrain the model often and add more parameters and stuff, it’s a totally different scale of things,” he says.
That is going to require a significant investment in hardware. “Current data centers and the infrastructure we have in place will not be able to cope with [the race of generative AI],” Bouchard says. “It’s too much.”
Data centers already account for around one percent of the world’s greenhouse gas emissions, according to the International Energy Agency. That is expected to rise as demand for cloud computing increases, but the companies running search have promised to reduce their net contribution to global heating.
“It’s definitely not as bad as transportation or the textile industry,” Gómez-Rodríguez says. “But [AI] can be a significant contributor to emissions.”
Microsoft has committed to becoming carbon negative by 2050. The company intends to buy 1.5 million metric tons worth of carbon credits this year. Google has committed to achieving net-zero emissions across its operations and value chain by 2030. OpenAI and Microsoft did not respond to requests for comment.
The environmental footprint and energy cost of integrating AI into search could be reduced by moving data centers onto cleaner energy sources, and by redesigning neural networks to become more efficient, reducing the so-called “inference time”—the amount of computing power required for an algorithm to work on new data.
“We have to work on how to reduce the inference time required for such big models,” says Nafise Sadat Moosavi, a lecturer in natural language processing at the University of Sheffield, who works on sustainability in natural language processing. “Now is a good time to focus on the efficiency aspect.”
Google spokesperson Jane Park tells WIRED that Google was initially releasing a version of Bard that was powered by a lighter-weight large language model.
“We have also published research detailing the energy costs of state-of-the-art language models, including an earlier and larger version of LaMDA,” says Park. “Our findings show that combining efficient models, processors, and data centers with clean energy sources can reduce the carbon footprint of a [machine learning] system by as much as 1,000 times.”
The question is whether it’s worth all the additional computing power and hassle for what could be, in the case of Google at least, minor gains in search accuracy. But Moosavi says that, while it’s important to focus on the amount of energy and carbon being generated by LLMs, there is a need for some perspective.
“It’s great that this actually works for end users,” she says. “Because previous large language models weren’t accessible to everybody.”
By Wired, March 26, 2023
