Planning your AI-ready infrastructure

Top 5 factors to consider

Primary & backup power

Generative AI models use graphics processing unit (GPU) chips which require 10–15 times the energy of a traditional CPU, because they have more transistors in the arithmetic logic units. This requires more compute, network, and storage infrastructure than any previous technology. Many generative AI models have billions of parameters and require fast and efficient data pipelines in their training phase, which can take months to complete, and these training phases then need to be repeated as new data is added.

Flexibility & scalability

Ramping up power usage requires new approaches to design and construction. Modular design is better equipped to cope with extra complexity and new datasets as AI and HPC arrays expand. A modular approach enables faster scalability and new cooling methods, as well as being quicker and cheaper to scale up. One report estimates that modular data centers are at least 60% faster to deploy and provide 13% or more cost savings than traditional data center power and cooling infrastructure.

Cooling considerations

Some computing hardware now enables power densities exceeding 100 kW/rack and the peak density in the data center could reach 150 kW/rack over the next few years. This increasing compression of computing power generates enormous heat which presents new challenges for data center cooling infrastructure.

Connectivity & latency

As new GPU chip designs emerge using 3D packaging and fiber-optic rather than copper connections, it is clear that rich low-latency data connectivity is the key to the future speed and efficiency of AI. The same principles apply at the macro or data center level. Data centers require high-bandwidth network connectivity to facilitate the efficient exchange of the large datasets necessary for training AI models. Advanced networking technologies such as high- speed optical fiber and low-latency inter-connected networks play a vital role in ensuring efficient data transfer and communication between servers. Modern AI applications require high-bandwidth, lossless, low-latency, scalable networks that can interconnect hundreds and thousands of GPUs at speeds as high as 400 or 800 Gbps.

Climate impact

Is an AI-driven world on a collision course with emission reduction targets? Not if users prioritize sustainability in their planning. Major commitments have been made by industry leaders to efficient use of resources like water and power, plus elimination or offsetting of emissions. All of these targets are fact-based and non-negotiable. As AI deployments use so much power both providers and users need to address these issues fully and transparently at the outset.