Retrofitting vs. New Builds: Navigating Liquid Cooling for AI Workloads

A recent analysis by gbc engineers highlights a pivotal challenge for data center operators: the strategic choice between retrofitting existing infrastructure for liquid cooling or investing in entirely new builds. This decision is no longer theoretical but a pressing reality driven by the escalating power demands of AI and High-Performance Computing (HPC) workloads. While most legacy data centers were designed for rack densities of 5-15 kW, modern AI servers, such as NVIDIA's H100 and H200 GPUs, can push a single 42U rack to an astonishing 60-100 kW. Such densities render traditional air cooling inefficient and often impractical above 30 kW per rack, necessitating a shift to more advanced thermal management solutions. This dilemma is critically important for cloud and DevOps professionals responsible for infrastructure planning and deployment. The ability to effectively cool high-density racks directly impacts an organization's capacity to host cutting-edge AI/ML training and inference, which is increasingly becoming a competitive differentiator. Incorrectly assessing the viability of a retrofit versus a new build can lead to substantial financial losses, project delays, and an inability to scale compute resources as demand for AI intensifies. The article emphasizes that the decision extends beyond merely selecting cooling equipment; it involves a complex evaluation of structural loads, pipe routing, facility water systems, power distribution, and the long-term maintenance implications. Furthermore, evolving regulations like the EU Energy Efficiency Directive are pushing for improved energy performance indicators (PUE) and water use efficiency, adding another layer of complexity to infrastructure decisions. The current discussion around liquid cooling is set against a broader, well-established trend of AI-driven transformation within data centers. The exponential growth in AI model complexity and data volumes has consistently driven up compute and storage requirements, leading to a relentless increase in power consumption per square foot. This has necessitated innovations across the data center stack, from power delivery and networking to, most critically, cooling. Liquid cooling, once a niche solution for supercomputers, has become a mainstream consideration as air-cooling limits are reached. This trend is also intertwined with the industry's broader push towards sustainability, where efficient cooling directly contributes to reducing energy consumption and environmental footprint, aligning with global net-zero ambitions. In practice, practitioners must initiate any liquid cooling project with a thorough, coordinated assessment involving structural, mechanical, electrical, and plumbing (MEP) experts from the outset. For rack densities ranging from 30 kW to 100 kW, Direct-to-Chip (DTC) cooling retrofits can be a viable option, provided the existing facility possesses adequate floor loading capacity and accessible plant room space. However, meticulous planning for coolant distribution unit (CDU) placement and pipe routing is essential to mitigate risks. For facilities targeting consistently higher densities or those with significant structural limitations, a purpose-built data center or a hybrid approach—creating dedicated liquid-cooled zones within an existing facility—may offer a more cost-effective and scalable long-term solution. Overlooking these critical considerations can result in costly reworks, operational inefficiencies, and a failure to meet the demanding performance requirements of modern AI workloads. The strategic choice should be informed by a holistic view of capital expenditure, operational costs, and future scalability needs, rather than a piecemeal equipment-centric approach.