Limiting AI’s Energy Footprint
UIC researchers develop software and optimize performance to reduce AI’s energy consumption
Artificial intelligence is a powerhouse for scientific discovery and businesses. Within a few short years, these powerful tools have become enmeshed in our daily lives, from personalized recommendations to customer service chatbots.
As the use of AI has skyrocketed, increasing demand at data centers is straining the power grid and depleting environmental resources. In addition to the millions of gallons of water used per day to cool data centers, energy is wasted through almost every exchange of information.
“In the past, we cared about performance; optimization was the one priority goal for the field,” said computer science Professor Zhiling Lan. “We optimized models without thinking about how to coordinate power management. But things have changed, power and energy are the most important things now and for the future.”
Researchers at UIC Engineering are at the forefront of this push to limit computing’s energy needs. Using their expertise, they are designing hardware that not only stores data but processes it locally, developing software that minimizes energy consumption and optimizes computing performance, and creating economic models to better manage workflow and limit downtime.
Managing power and energy waste
As the world increasingly relies on AI, the power problem will only continue to grow. OpenAI’s GPT-4 model consumed 50-gigawatt-hours of energy to train, enough energy to power 5 million homes in the U.S. for a day.
A new projection published by Lawrence Berkeley National Laboratory predicts that by 2028, more than half of the electricity going to data centers will be used to power AI, which is poised to consume as much electricity as 22% of all US households.
To help curb this, UIC Engineeirng professors are starting on their home turf. Lan has a joint appointment at Argonne National Laboratory, which operates several supercomputers, including the exascale Aurora computer, which can perform over a quintillion calculations per second. While a home computer has one central processing unit, or CPU, supercomputers have tens of thousands of connected nodes, each holding multiple CPUs and graphics processing units (GPUs), which are designed to handle complex visual and mathematical calculations.
Last year, Lan and Michael E. Papka, the Collegiate Warren S. McCulloch Professor of Computer Science and the director of the Argonne Leadership Computing Facility, were motivated to address two critical gaps in high-performance computing (HPC): power waste and dynamic power management.
During computing, not all the CPUs and GPUs in a supercomputer may be in use. While they can’t just be shut down, they can be placed in a higher-power state or be idled. But even an idle GPU consumes 60 watts of power. Lan and Papka conducted a field study of supercomputers and quantified the power waste among those HPCs at a staggering 50%.
1
We’re rapidly hitting the point where throwing more resources at the problem is unsustainable, so it’s more appealing to find the solutions that let you operate things at a higher utilization.
| Associate Professor of Computer Science
2.2
In response, Lan and Papka developed an adaptive runtime system that dynamically predicts resource demands and allows applications to minimize power waste while maximizing performance. Their software uses AI to make recommendations to move power to where it is most needed and limits idle time through better application management.
“Everyone wants to build up their data center so that they can run AI,” Lan said. “But the power grid cannot handle the energy demands, and there are environmental concerns to building more data centers without first addressing efficiency.”
Who gets what: managing computing resources
When demand outstrips supply, as is the case with AI tools and available computing resources, mechanisms are needed to manage the allocation of resources. Ian Kash, an associate professor of computer science who works at the intersection of economics and computer science, is developing some market-based solutions in collaboration with Lan and Papka.
Access to computing resources can vary: the time a user can book might depend on availability and on other jobs that take precedence. Big government supercomputers have a similar issue; scientists are granted allocations of time, but don’t know exactly when they will receive that time and are competing for finite resources.
This resource allocation problem is similar to what the airline industry faces daily. An airline’s goal is to make sure all seats are filled and maximize profits based on demand, cutting prices if seats are selling too slowly or raising prices when demand is high.
UIC graduate students Andrea Giarduz and Matteo Del Grossi applied models similar to those used by airlines to the computational demand problem. Giarduz examined the promise of this approach for online cloud scheduling in comparison to existing pricing schemes such as on demand computing, volume discounts, spot use, and reserved instances. Del Grossi introduced a secondary preemptive market that would optimize the use of computational resources by allowing access to resources, knowing potential interruptions may occur.
Supercomputing sites are starting to use similar tactics, implementing a small discount if a customer is willing to make their workload preemptible if something more pressing comes in.
Kash, along with three researchers at University of Chicago, also created an algorithm that continuously assigns jobs to available servers to maximize the value of jobs processed.
“That’s where markets shine, in figuring out who should be left out when we’re running low on resources,” Kash said. “We’re rapidly hitting the point where throwing more resources at the problem is unsustainable, so it’s more appealing to find the solutions that let you operate things at a higher utilization. You have to pay the cost somewhere.”
For inspiration, look to the brain
Amit R. Trivedi, an associate professor of electrical and computer engineering, is turning to the human brain to improve computer hardware. Computer chips have long served as memory, transmitting data back and forth to CPUs and GPUs for computation. But AI models work very differently from other algorithms, with each model containing billions or trillions of parameters that require data to be transferred back and forth between multiple memories for computation.
For this work, the brain’s architecture might make a better model. “In our brains, storage and compute functions are all fused within each other, we store information in synapses and compute with neurons,” Trivedi said. “They are all in a connected mesh.”
This evolutionary advantage is clear: an AI model may need to be trained on thousands of images of cars to recognize one, but a four-year-old only needs to see a handful of cars before she can do the same. And a human brain only takes about 20 watts of power, compared to an NVIDIA GPU that takes kilowatts of power to achieve the same thing.
Trivedi’s work with compute-in-memory chips, in partnership with Intel, not only eliminates some of the back-and-forth required with traditional chips, but speeds the process, enabling more real-time decision making—key to self-driving cars or autonomous robots.
Trivedi is also collaborating with a group of material scientists and neuroscientists at Northwestern University to replicate mechanisms inside the brain. The group is studying the cerebellum, the back-of-the-brain workhorse that coordinates movement and balance, helps with cognitive functions like attention and language, and assists with decision making and emotional regulation.
The brain is constantly receiving and ignoring massive amounts of sensory signals but remains vigilant to unanticipated events. For example, the cerebellum recognizes when you are about to fall, and somehow knows how to keep you on balance, quickly countering forces. Machine learning requires a lot of data training to do the same thing.
And while a brain is three dimensional, chips must be two dimensional, so Trivedi and his colleagues are working to replicate the mechanisms that allow for this within the constraints of chips.
quote
In our brains, storage and compute functions are all fused within each other, we store information in synapses and compute with neurons. They are all in a connected mesh.
| Associate Professor of Electrical and Computer Engineering
All hands on deck
The easy answer is to continue to build more data centers, but these centers will continue to dominate the power grid, impact the environment, and cause issues we haven’t begun to recognize. Finding better ways to compute— through collaboration with researchers from other fields and universities, and with national labs and industry—are some of the ways UIC researchers are hoping to positively impact AI.