Power & Cooling Design for an L40S Cluster: The 4.6kW-per-Node Utility Ledger

A fully loaded 8×L40S node is a 4.6kW space heater; 48 of them are a ~220kW heat source. Power and cooling are not something to think about after the racks are in place — they decide rack density, the power chain and the cooling method. This article is the utility ledger of a 384-GPU L40S IB cluster.

1. The Per-Node Power Ledger: Do the Math Before Buying Power

Component	Power	Qty	Subtotal
CPU (8468V)	330W	2	660W
GPU (L40S)	350W	8	2800W
Memory/disks/NICs/motherboard	≈ 400-500W	—	≈ 450W
Full-load total			≈ 4.6kW

Design at 6kW/node redundancy — not waste, but headroom for the power chain, cooling and transients: when 8 GPUs peak simultaneously with memory bandwidth maxed, instantaneous draw exceeds steady state.

2. Rack and Facility Budget

Dimension	Calculation	Conclusion
Per-rack density	At 6kW/node × density	8-10 nodes/rack (depends on cooling); avoid 12×4.6kW in one rack exceeding power capacity
Total power	48 × 6kW ≈ 288kW	Reserve capacity accordingly, with redundancy
Cooling	~288kW heat needs equivalent cooling	Liquid or high-density air cooling per rack heat density
Network gear	Leaf/Spine + storage switches	Count separately; QM switch draw is not negligible

Judgment: the most common mid-size cluster mistake is over-densifying racks — cramming 12×4.6kW machines into a plain 42U rack trips power and cooling alarms at once, ending in derated operation.

3. Power Chain and Power Sequencing

• Power-on order: management plane (out-of-band IPMI/mgmt switches) → storage/network → compute; power-off in reverse. Avoids compute nodes coming up while network/storage are unavailable and hanging;
• Group-by-group power-on: do not power all 48 at once — go rack-by-rack/group-by-group to avoid inrush hitting the power chain;
• Temperature monitoring: GPU temps, intake/exhaust air temps monitored; alert thresholds set at delivery, not after an incident.

4. Verification & Acceptance

# Full-load verification: watch 8-GPU power and temp (run HPL/NCCL at full load)
nvidia-smi --query-gpu=power.draw,temperature.gpu --format=csv
# Power verification: 30 min full load, no derating (P-state stays put)

Acceptance criteria:

1. 30 min full load: GPU temps within vendor spec, no derating;
2. Measured per-node power within the 6kW design;
3. One power sequencing drill, correct order, no hangs;
4. Temperature alert thresholds configured and trigger-tested.

5. Delivery Checklist

1. Power ledger: measured 4.6kW full load; supply designed at 6kW/node redundancy;
2. Rack density and power/cooling budget documented;
3. Power sequencing documented + one drill;
4. Temperature/power monitoring and alerts configured;
5. Full-load verification archived (temp, power, no derating).

---

Based on field measurements from a 384-GPU L40S IB cluster delivery; exact versions and commands follow on-site validation.