AI data centers generate far more heat than traditional cloud facilities because of dense clusters of high-power GPUs and accelerators. A single modern AI rack can draw 100-200+ kW (and rising toward higher levels), compared with the 5-20 kW typical of older air-cooled racks. This heat density makes traditional cooling inadequate and has driven a rapid shift toward more efficient techniques.
Cooling removes heat from chips so they can run at full performance without thermal throttling. The main methods used in AI facilities in 2026 fall into three categories: air cooling, direct-to-chip liquid cooling, and immersion cooling. Many sites use hybrid approaches.
Companion pieces from our AI-infrastructure series: How Much Water AI Uses Per Prompt and How Much Electricity AI Uses Per Prompt.
1. Air Cooling (Traditional Method)
Air cooling remains common for lower-density or mixed workloads but is reaching its practical limits for high-end AI training clusters.
How it works
- Computer Room Air Conditioners (CRAC) or Computer Room Air Handlers (CRAH) cool and circulate air.
- Raised floors, hot-aisle/cold-aisle containment and rear-door heat exchangers help direct airflow.
- Fans inside servers and large facility fans move heat away from components to chilled-water coils or outside air.
Strengths
- Simple, familiar and lower upfront cost for existing facilities.
- Works well for racks under ~15-25 kW.
Limitations for AI
- Air is a poor heat conductor. High-power GPUs (700-1,000+ W each) generate more heat than air can efficiently remove in a standard rack.
- High fan power consumption increases overall energy use (PUE often 1.3-1.8).
- Cannot support the densest AI racks (e.g., NVIDIA GB200 NVL72 or similar systems that exceed 100-130 kW).
Many operators still use air cooling (or hybrid air + liquid) for networking equipment, storage or lower-density inference servers.
2. Direct-to-Chip Liquid Cooling (Most Common for AI)
This is the dominant approach for modern high-density AI “factories.”
How it works
- Cold plates (metal plates with internal channels) are mounted directly on the hottest components — primarily GPUs, CPUs, and sometimes networking ASICs.
- A dielectric or water-based coolant (often a water-glycol mix) flows through the cold plates, absorbing heat at the source.
- The warmed liquid travels to a Coolant Distribution Unit (CDU), which transfers the heat to a facility water loop.
- The facility loop then rejects heat outdoors via dry coolers, cooling towers or chillers.
Key advantages
- Captures 70-96% of rack heat in liquid (far more efficient than air).
- Supports rack densities of 50-200+ kW.
- Lowers chip temperatures, allowing sustained peak performance.
- Enables higher coolant temperatures (up to 45°C / 113°F in newer designs). Reduces or eliminates the need for energy-intensive chillers in many climates and can cut water use dramatically when paired with dry coolers.
- Typical PUE improvements to the 1.05-1.20 range.
Variants
- Single-phase — coolant stays liquid.
- Two-phase — coolant boils at the chip for even better heat transfer.
Direct-to-chip systems are now standard for NVIDIA Blackwell, Rubin and similar platforms. Many hyperscalers deploy them in hybrid setups: liquid for AI accelerators and air for remaining components.
3. Immersion Cooling
Servers are fully submerged in a dielectric (non-conductive) fluid.
How it works
- Entire servers sit in sealed tanks filled with specialized fluid.
- Heat transfers directly from components to the fluid.
- Single-phase — fluid remains liquid and is pumped to heat exchangers.
- Two-phase — fluid boils at the hot surfaces; vapor rises, condenses and returns (highly efficient heat transfer).
Strengths
- Excellent for extreme densities (100-250+ kW per tank or rack equivalent).
- Very low PUE (often 1.01-1.08) because server fans are eliminated.
- Uniform cooling across all components.
- Quiet operation.
Trade-offs
- Higher capital cost and more complex maintenance (hardware must be removed from fluid for some service).
- Requires purpose-built infrastructure rather than standard racks.
- Fluid management and compatibility considerations.
Immersion is used in specialized high-density AI and HPC deployments but is less common than direct-to-chip for mainstream hyperscale AI clusters in 2026.
Hybrid and Supporting Approaches
Most real-world AI data centers use hybrids:
- Direct-to-chip liquid for GPUs + air or rear-door heat exchangers for residual heat and non-GPU gear.
- Facility-level heat rejection via cooling towers (evaporative, water-intensive but efficient), dry coolers (air-based, near-zero water use), or free cooling (using outside air when climate allows).
Emerging innovations include advanced two-phase systems, on-die microchannel cooling and nuclear-inspired bubble-enhanced phase-change methods that further improve efficiency.
Efficiency Metrics That Matter
- PUE (Power Usage Effectiveness) — closer to 1.0 is better. Liquid and immersion systems achieve significantly lower PUE than pure air cooling.
- Water use — traditional evaporative cooling towers consume large volumes of water. Higher-temperature liquid loops + dry coolers can reduce this dramatically or eliminate it in favorable climates.
- Chip temperature — lower junction temperatures improve performance and reliability.
Quick Comparison Table
| Method | Rack Density | Typical PUE | Water Use | Best For |
|---|---|---|---|---|
| Air (CRAC/CRAH) | 5-25 kW | 1.3-1.8 | Moderate | Networking, storage, legacy servers |
| Direct-to-chip liquid | 50-200+ kW | 1.05-1.20 | Low-moderate | Modern AI training clusters (Blackwell, Rubin) |
| Immersion (single-phase) | 100-250+ kW | 1.03-1.10 | Very low | Specialty high-density HPC/AI |
| Immersion (two-phase) | 100-250+ kW | 1.01-1.08 | Very low | Extreme density, silent operation |
Why This Matters for AI
Cooling can account for a substantial portion of a data center's total electricity use. Better methods lower energy and water footprints, enable denser (and therefore more powerful) clusters in the same footprint, and support the rapid growth of AI infrastructure. As rack powers continue climbing, liquid cooling — especially direct-to-chip — has become essential rather than optional for frontier AI workloads.
Why This Matters for the Indian Diaspora and Indian Tech
- India's data-center boom. Mumbai, Chennai, Hyderabad, Pune and Bengaluru are among the fastest-growing global data-center hubs. Understanding cooling methods matters for infrastructure investment decisions and site planning.
- Career signal. Skills in data-center engineering, thermal design, direct-to-chip liquid systems, PUE optimization and facility management are increasingly valuable for Indian tech professionals working at hyperscalers and colocation providers.
- Investment lens. NRI investors in Indian data-center REITs (CtrlS, Yotta, Nxtra, Sify, ST Telemedia, NTT-Netmagic) benefit from understanding the CapEx and OpEx implications of liquid vs air cooling.
- Sustainability policy. Water-stressed Indian cities like Chennai are pushing operators toward liquid + dry cooler designs to reduce water consumption — a material siting and permitting question.
Frequently Asked Questions
Why can't air cooling handle modern AI racks?
Air is a poor heat conductor. Modern GPUs (700-1,000+ W each) in dense racks (100-200+ kW total) generate more heat than air can efficiently remove within the physical rack volume. Direct-to-chip liquid cooling captures 70-96% of the heat directly at source.
What is a PUE and what's a good target?
PUE (Power Usage Effectiveness) is total facility power divided by IT equipment power. 1.0 is ideal (all energy goes to IT). Air-cooled facilities typically 1.3-1.8; direct-to-chip liquid 1.05-1.20; immersion 1.01-1.10.
Is immersion cooling always better than direct-to-chip?
Not always. Immersion has lower PUE and higher density but comes with capital cost, maintenance complexity and fluid-management overhead. Direct-to-chip is the mainstream hyperscale choice; immersion serves specialty extreme-density and silent-operation niches.
How does cooling affect data-center water use?
Traditional evaporative cooling towers consume large volumes of water. Modern high-temperature liquid loops (up to 45°C coolant) paired with dry coolers can dramatically reduce or eliminate water use, especially in favorable climates.
What's next after direct-to-chip?
Advanced two-phase systems, on-die microchannel cooling and bubble-enhanced phase-change methods (some inspired by nuclear-reactor thermal designs) are actively researched. Frontier chips like NVIDIA Rubin and successors are being designed with cooling-first architectures in mind.
Should Indian data centers move to liquid cooling?
Increasingly yes for AI-focused facilities. Water-stressed Indian metros (Chennai in particular) already favor liquid + dry cooler designs over evaporative towers. Colocation providers building for AI workloads are actively deploying direct-to-chip.
Summary: Choose the Right Cooling for the Workload
- Air cooling handles lighter loads (networking, storage, low-density inference).
- Direct-to-chip liquid cooling is the workhorse for today's AI factories (50-200+ kW racks, PUE ~1.05-1.20).
- Immersion cooling serves the highest-density and silent-operation niches (100-250+ kW, PUE 1.01-1.10).
- Hybrid designs dominate the transition period as facilities balance legacy equipment with next-generation accelerators.
Related NRI Globe Coverage
- How Much Water AI Uses Per Prompt (Companion)
- How Much Electricity AI Uses Per Prompt (Companion)
- Generative AI Terminology 2026 — Complete Glossary
- Best AI Tools for NRIs 2026
- Best Tech Jobs for NRIs 2026
- Anthropic CEO on AI Control
In short: air cooling handles lighter loads, direct-to-chip liquid cooling is the workhorse for today's AI factories, and immersion serves the highest-density niches. Hybrid designs dominate the transition period as facilities balance legacy equipment with next-generation accelerators.
Disclaimer: Rack power densities, PUE ranges, and specific product capabilities cited (NVIDIA GB200 NVL72, Blackwell, Rubin) reflect widely reported industry figures as of 2026 and continue to evolve rapidly. Always verify current specifications with the OEM datasheet before making infrastructure decisions. NRI Globe provides journalism and general information only.