AI boom forces data centers to prioritize efficiency

The rapid expansion of AI infrastructure is colliding with physical limitations in power, cooling, and utility capacity, forcing data center operators to rethink long-standing assumptions about workload placement and infrastructure design. As rack densities rise and AI clusters grow, energy efficiency has moved from a sustainability concern to a core operational priority, according to HPE’s principal technologist for sustainable transformation, Andrew DesRochers.

This shift reflects broader industry pressures, including rising energy costs, regional power constraints, and growing scrutiny of water consumption tied to new developments. Operators are now evaluating efficiency across IT and facility systems, rather than treating energy as an afterthought handled by facilities teams. The focus has narrowed to execution, with organizations linking efficiency directly to business outcomes and return on investment for AI workloads.

Power and cooling emerge as gating factors

Utility interconnection timelines and power distribution equipment are increasingly shaping infrastructure decisions. In some regions, operators are exploring high-voltage DC and other alternatives to meet future power demands, while cooling challenges are prompting a reevaluation of facility-level infrastructure. DesRochers noted that water availability is becoming a key consideration in site selection, particularly in resource-constrained areas, with some organizations willing to accept higher latency to leverage cooler climates or more efficient facilities.

Rack densities are rising, with liquid cooling no longer an edge case but a necessity for many high-performance deployments. However, DesRochers emphasized that cooling efficiency extends beyond compute hardware to include networking and storage systems, as well as broader facility operations. Direct liquid cooling, often assumed to be the primary source of water consumption, is typically self-contained, but operators must evaluate the full system to understand resource use.

Enterprise AI workloads differ from training clusters

A critical distinction is emerging between large-scale AI training facilities and typical enterprise AI deployments. Most enterprises are not training foundation models from scratch but instead deploying inference workloads or using smaller, distilled models. These workloads have different infrastructure requirements, with lower power and cooling demands than the massive training clusters dominating public discussions. DesRochers cautioned against treating all AI workloads as identical, noting that their efficiency profiles vary widely.

Utilization has become a major focus, with organizations discovering that simple changes—such as adjusting performance modes or upgrading to more efficient fans—can significantly reduce energy consumption without impacting outcomes. Measurement and analytics are now essential for identifying optimization opportunities, as visibility into power consumption and cooling requirements enables better decision-making.

Operational behavior shifts toward deliberate adoption

Early AI deployments were often driven by competitive pressure, with organizations pursuing generative AI because "everyone else was." Today, operators are adopting a more deliberate approach, questioning whether AI is the right tool for specific problems and which deployment models align with business objectives. This shift reflects a broader trend toward practical applications and measurable outcomes, with efficiency embedded into AI strategies from the outset.

DesRochers highlighted that adoption and employee understanding of AI’s role in workflows are critical to achieving meaningful returns on investment. Organizations are reassessing strategies based on lessons learned from early experiments, with a growing recognition that efficiency must be a default design consideration rather than a topic requiring constant education.

What to watch

Over the next two years, efficiency is expected to become a standard part of infrastructure design, deployment, and operations. Waterless cooling technologies, which historically carried a cost premium, are becoming more economically viable as communities and regulators tighten scrutiny on water consumption. Meanwhile, utility constraints and power availability will likely continue to influence site selection, with operators balancing latency, climate, and resource optimization in their decisions.

The industry is also grappling with the need for better data and analytics to drive efficiency gains. Without granular visibility into power consumption, cooling requirements, and system utilization, organizations will struggle to identify optimization opportunities or justify investments in newer, more efficient infrastructure.