AI Cluster Architecture Design: From Requirements to a Layered Blueprint

Note: This is a review article (no first-hand experimental data), based on public cluster-architecture practice.

Experiment Objective

Cluster architecture is the "master plan" of AI-HPC: capacity planning (ch02), network design (ch03), storage (ch04) are parts; this article assembles them into a machine. Goal: a complete requirements-to-blueprint process — requirements derivation (scale/workload/availability) → four-plane layering (compute/storage/network/management) → cross-plane bandwidth budgeting → architecture documents and a design checklist. After reading you can answer: how to design a cluster from requirements, how to layer the planes, how to budget bandwidth, and what an architecture document contains.

Code Walkthrough

Part 1: Deriving scale — from business goals to cluster size

def size_cluster(workloads, target, concurrency):
    # Training-load-first derivation (estimated):
    # 1) per-GPU compute × cards = total compute; back out cards from
    #    "target model scale / training period" (compute-cost formula, ch10)
    # 2) concurrent projects × per-project cards = parallelism need (multi-team)
    # 3) inference/scientific-computing load share → add-on demand (ch02 planning)
    # 4) redundancy: N+1 power/network, fault-domain design
    total_gpus = model_compute(target) / (gpu_flops * days) + concurrency_gpus
    return plan(total_gpus)

Key points:

1. Scale is derived from "target model × training period": not "how much compute to buy" but "how many cards for a given model and period" — ch10's compute-cost formula is the basis (estimated).
2. Parallel demand is separate: multi-team concurrency = projects × per-project cards, added to training demand — capacity planning (ch02) feeds architecture directly.
3. Redundancy is a design constraint, not optional: N+1 power/network, fault domains — layer redundancy on after scale derivation.

Part 2: Four-plane layering and bandwidth budgeting

# Four planes (linking each chapter):
#   Compute: GPU nodes + high-speed interconnect (ch03 IB/RoCE)
#   Storage: shared storage (ch04 parallel filesystem) + data-loading network
#   Network: compute/storage/management separation (ch03 plane decoupling)
#   Management: out-of-band (BMC/IPMI) + monitoring (ch02 DCGM)
# Bandwidth budgeting (empirical, estimated):
#   Compute net: 1:1 non-blocking (GPU communication), 400G at 10K scale
#   Storage net: by "checkpoint write window" (ch04: dump GPU memory in minutes)
#   Management net: low bandwidth (out-of-band + monitoring telemetry)

Key points:

• The four planes are the skeleton: compute/storage/network/management each has a role; chapter knowledge aligns here — architecture is "assembling the parts into a machine."
• Bandwidth follows from the critical path: compute net by AllReduce non-blocking, storage net by checkpoint window (ch04), management net low — every link has a business reason.
• Plane separation is the 10K-scale premise: without separation, traffic interferes (checkpoints drag training communication) — ch03's decoupling lands here.

Principles

Layering and chapter alignment

Requirements (model/period/concurrency/redundancy)
  └── Compute plane: GPU nodes (ch01 hardware) + interconnect (ch03 network)
  └── Storage plane: parallel filesystem (ch04 storage) + data pipelines (ch13 data)
  └── Network plane: three-network separation (ch03) + telemetry (ch02 DCGM/monitoring)
  └── Management plane: provisioning (ch05) + scheduling (ch06 cloud-native) + governance (ch02)

Core insight: cluster architecture is not "buying a pile of devices" but requirements-driven layered design — each layer is supported by the one below and serves the one above; chapters are in-layer details, architecture fixes the inter-layer relationships.

Key design decisions checklist

Decision	Question	Linked chapter
scale	how to derive cards/compute	ch02 planning, ch10 compute cost
interconnect	IB or RoCE, oversubscription	ch03 selection
storage	capacity/bandwidth/checkpoint window	ch04 storage
redundancy	N+1, fault domains, HA	ch05 PXE HA, ch03 network HA
scheduling	queues/partitions/QoS	ch06 cloud-native, ch07 heterogeneous

Architecture document = decisions + reasons: each decision written as "what + why + numerical basis" makes the document reviewable and evolvable.

Results

(Estimated, not measured)

Scale	Compute plane	Storage bandwidth	Network oversubscription
1K	400G interconnect	hundreds of GB/s	compute net 1:1
10K	400G layered (SU units)	TB-class	compute net 1:1 non-blocking

Engineering Takeaways

1. Requirements first: scale derived from "target model × period"; capacity planning (ch02) feeds architecture; redundancy is a constraint.
2. Four-plane layering: compute/storage/network/management each has a role; chapter knowledge aligns at the architecture layer — architecture "assembles the machine," not "buys parts."
3. Bandwidth from the critical path: compute by AllReduce, storage by checkpoint window, management low — every link justified.
4. Document decisions and reasons: architecture doc = "what + why + numbers" per decision; reviewable and evolvable.

Next article preview: the master plan settled; back to the framework layer — "Model Parallelism Explained: Tensor, Pipeline, and Sequence Parallelism."