● Covered by 1 source · 1 reportMedium impact1 neutral
Software Optimization Offers Significant Energy Savings for AI Data Centers
Key points
- AI data centers' energy demand is projected to reach 945 TWh by 2030.
- Software optimization can reduce AI training energy by up to 30%.
- Lower precision formats like FP8 consume less energy for inference.
- Hardware changes are slow and costly; software offers faster efficiency gains.
Rising Energy Demands of AI
The energy consumption of data centers, particularly for AI, is rapidly increasing. Projections indicate that by 2030, AI will demand 945 TWh of electricity, equivalent to Japan's current total electricity usage. The industry has primarily focused on hardware solutions, such as more efficient GPUs and securing additional power supply, to address this energy crunch.
Limitations of Hardware-Centric Approach
Despite efforts, the average power usage effectiveness (PUE) of data centers has remained largely unchanged for six years. PUE measures cooling and power system waste but does not account for the efficiency of the software running on servers. Servers contribute approximately 60% of a data center's electricity demand, suggesting that software optimization could yield significant energy savings.
Software as a Solution
Jae-Won Chung, a PhD candidate at the University of Michigan and researcher with ML.Energy, states that software and algorithms can meaningfully contribute to power and energy savings. He views computing systems as a stack, with hardware at the base and systems software, algorithms, and applications above. While hardware is slow and costly to replace, the upper layers offer easier and more impactful opportunities for efficiency gains.
Demonstrated Software Efficiencies
ML.Energy's tests on the Alibaba Qwen 3 235B A22B Thinking model showed that running inference in FP8, a lower-precision format, consumed one-third less energy than bfloat16 versions for problem-solving tasks. Chung's Perseus training optimizer reduced training energy by up to 30% without affecting throughput or requiring hardware changes, by identifying and slowing less busy parts of large-model training jobs.
Industry Adoption
Major GPU firms are also exploring similar software-based optimizations. Nvidia's Blackwell power profiles, for instance, fine-tune GPU operations to improve efficiency. This indicates a growing recognition within the industry of software's role in addressing the energy demands of AI.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
Researchers suggest software and algorithmic improvements can substantially reduce the energy consumption of AI data centers, which are projected to use as much electricity as Japan by 2030. Focusing on software efficiency, rather than solely hardware, could cut energy use by up to 30% without impacting performance. This approach addresses the increasing energy demands of AI workloads.