Liquid-Cooled Data Center Design: From Air-Cooling Limits to Cold Plates
Abstract: At H100's 700 W per card and 100+ kW per rack, "should we go liquid cooling" has become "liquid cooling is mandatory." This article clarifies where the air-cooling limit sits, how to choose the liquid-cooling approach, key components, and acceptance/operations — a directly applicable design method for cluster delivery.
1. Why Liquid Cooling Is Mandatory: The Physics of Air Cooling
Air-cooling capacity ≈ ambient environment (supply temp/air speed) × rack surface area
Typical air-cooling limit: 50~60 kW/rack (40U rack, 18–22°C supply)| Generation | Per-card TDP | 8-GPU node | Full rack (6–8 nodes) | Air cooling feasible? |
|---|---|---|---|---|
| A100 | 400 W | 3.2 kW | ~20 kW | ✅ barely |
| H100 | 700 W | 5.6 kW | 40–50 kW | ⚠️ at the limit (needs high airflow) |
| B200+ | ~1000 W | 8+ kW | 80–120 kW | ❌ liquid mandatory |
| Rack-scale Scale-Up | — | — | 100 kW+ normal | ❌ liquid mandatory |
Decision formula: rack power density > 55 kW/rack → liquid cooling is a precondition, not an option.
2. Liquid-Cooling Approach: Cold Plate vs Immersion
| Dimension | Cold Plate (Direct-to-Chip) | Immersion |
|---|---|---|
| Principle | Coolant contacts CPU/GPU via cold plates | Whole server immersed in coolant |
| Heat-transfer efficiency | High (precise chip-level cooling) | Highest (all components cooled) |
| Retrofit cost | Moderate (standard racks compatible) | High (special tanks/fluid) |
| Maintenance | Close to traditional | Complex (remove whole server) |
| Best for | Mainstream AI/HPC choice | Ultra-high density / supercomputing |
Engineering conclusion: cold-plate is the mainstream — covers 100–150 kW/rack, near-traditional maintenance, best cost-to-deliverability. Immersion is reserved for >150 kW/rack ultra-high density.
3. Cold-Plate Key Components
3.1 Primary and Secondary Loops
Outdoor → cooling tower/dry cooler (primary loop, ~35°C out)
↓
CDU (coolant distribution unit: heat exchange + pumps + water quality)
↓
In-rack manifold (headers) → quick connects → server cold plates (secondary loop, ~25°C in)3.2 Component Selection Essentials
| Component | Key parameters | Note |
|---|---|---|
| CDU | Heat-exchange capacity ≥ rack total × 1.25 (redundancy) | Primary/backup pumps, N+1 redundancy |
| Quick connects | Dry-break (no-drip) design | 80% of leaks originate at quick connects; use certified models |
| Coolant | Deionized water (DIW) or propylene-glycol mix | DIW low conductivity but needs corrosion control; mixes resist freezing but transfer slightly less |
| Manifold | One per rack with flow meters and temp sensors | Flow/delta-T into monitoring |
| Leak detection | Rope/point sensors | ≥1 path per rack, into alerting with auto-shutoff |
3.3 Primary Loop Design
- Redundancy: CDU N+1, dual-power pump stations, cold source (cooling towers) 2N or with a thermal storage tank.
- Water quality: primary loop chemical dosing/filtration/biocide; secondary loop DIW conductivity monitoring (< 1 µS/cm).
- Temperature strategy: secondary-loop supply at 25°C to start (higher water temps allowed while GPU junction stays < 85°C); higher water temps = better PUE.
4. Coexisting with Air Cooling: Hybrid Facilities
AI clusters usually mix liquid-cooled GPU nodes + air-cooled network/storage nodes:
- IB switches are air-cooled: switch racks are low-density (< 20 kW/rack) and air-cooling suffices, but they must not sit in the "hot aisle" of liquid-cooled racks.
- Layout: separate liquid-cooled (GPU nodes) and air-cooled (network/management/storage) zones; seal cold/hot aisles of liquid racks to avoid mixing.
- Fresh air/humidity: 40–60%RH; condensation risk lives at quick connects and cold-plate surfaces — dew-point monitoring is mandatory.
5. Acceptance & Operations: Liquid Cooling Is Not Install-and-Forget
5.1 Four-Step Acceptance
1. Per-rack flow/delta-T acceptance: delta-T within spec at design flow (usually 10–15°C)
2. Load stress test: 24h full load, GPU junction < 85°C, no CDU alarms
3. Leak drill: simulated leak (water injection test), verify detection + auto-shutoff
4. Redundancy test: primary pump failover without interrupting training (< 5s)5.2 Operations Essentials
- Leak patrols: quarterly quick-connect torque recheck + visual inspection; leak sensor self-tests.
- Water quality: monthly DIW conductivity/microbiology; resin replacement per vendor cycle.
- Maintenance windows: liquid-cooling maintenance (CDU cleaning, filter changes) combined with hardware maintenance to reduce downtime.
- Failure plans: how long the secondary loop can ride on natural circulation/emergency cooling after CDU failure — written into the runbook (typically ≥ 15 minutes emergency window).
6. Design Parameter Quick Reference
| Parameter | Recommended | Notes |
|---|---|---|
| Secondary-loop supply temp | 25–32°C | Higher = better PUE; GPU junction is the ceiling |
| Rack liquid-cooling capacity | ≥ 100 kW/rack | Reserve for next GPU generation |
| CDU redundancy | N+1 | Both heat exchange and pumps |
| Leak sensors | ≥1 path per rack + focused at quick connects | Auto-shutoff power/pumps |
| Coolant conductivity | < 1 µS/cm | DIW standard |
| Emergency window | ≥ 15 minutes | Safe training shutdown time after CDU failure |
Further reading: 10K-GPU Cluster Hardware Selection & Acceptance Checklist (power and cooling selection dimensions), GPU Drivers & Lifecycle Management (hardware lifecycle support).