Collective Topology Quantified: ring, tree, and rail

"How the cards are wired" inside a supernode directly decides the communication ledger: for the same All-Reduce, ring uses bandwidth fully but takes N steps, tree has only logN depth but relay overhead, and rail lets same-indexed cards reach each other in one hop. This article quantifies the three topologies and compares them. Method framework references the open-source book Understanding AI Infra (Apache-2.0); formulas and examples are derived by this site and are reproducible.

1. ring: Best Bandwidth, Most Steps

ring chains N cards; each card talks only to its neighbors (NCCL's default):

$$T_{ring} \approx \frac{2(N-1)}{N} \times \frac{Msg}{BW}, \quad busbw = \frac{2(N-1)}{N} \times BW$$

• Pros: every link participates, highest bandwidth utilization (approaches 2×BW);
• Cons: latency grows linearly with N (N-1 steps); poor for small messages.

Fits: large messages (GB-scale gradient all-reduce) — the training default.

2. tree: logN Depth, Relay Cost

Tree all-reduce reduces/broadcasts level by level (with SHARP doing in-switch reduction):

$$T_{tree} \approx O(\log N) \times (Msg/BW), \quad but\ effective\_BW = \frac{BW}{fanout}$$

• Pros: latency grows logarithmically with N, far better than ring at large N;
• Cons: root/relay bandwidth is the bottleneck; the larger the fanout, the lower the effective BW; SHARP pushes reduction into switches so the switch pays the relay cost, sharply raising effective bandwidth.

Takeaway: large N + large message → tree/SHARP beats ring; at small N, ring's simplicity is more robust.

3. rail: The "One-Hop Ledger" Set by Topology

Rail optimization is not an algorithm but physical topology: connect the i-th GPU of N nodes to the same rail switch, so same-indexed GPU collectives finish in one hop:

$$T_{rail\_same} = \frac{Msg}{BW_{leaf}}, \quad T_{rail\_cross} = \frac{Msg}{BW_{leaf}} + \frac{Msg}{BW_{spine}} \times (\text{cross-rail count})$$

Scope	Path	Bandwidth ledger
Same rail (same index)	node → Leaf → node	1 hop, full leaf BW
Cross rail (different index)	node → Leaf → Spine → Leaf → node	2 hops, one extra spine pass
Full-card All-Reduce	mixed (ring crosses rails)	capped by slowest link

This is the quantitative origin of the Rail Topology article's "same-index comm is optimal": if collectives align to rails (same-indexed cards do the same communication group), cross-rail traffic is minimized and effective bandwidth is highest.

4. Three Topologies Compared

Topology	Bandwidth model	Latency vs N	Fits
ring	2(N-1)/N × BW	linear	large messages, training gradients
tree (incl. SHARP)	logN depth, fanout discount	logarithmic	large N & message, in-network reduction available
rail (topology)	same-rail 1-hop full BW	depends on traffic distribution	workloads with index-aligned collectives

Combo: modern clusters stack "ring algorithm + rail topology + SHARP offload" — algorithm guarantees bandwidth use, topology guarantees locality, switches offload relaying — which is the quantitative rationale behind 4 rails + 4 spines + adaptive routing in the L40S plan.

5. Reproducible Python

def comm_topology_compare(msg_gb, bw_gbps, n, fanout=4):
    """ring vs tree time (seconds, simplified)"""
    t_ring = 2 * (n - 1) / n * msg_gb / (bw_gbps / 8 * (2 * (n - 1) / n))
    t_tree = 2 * msg_gb / (bw_gbps / 8 / fanout)      # 2 passes × relay effective BW
    return t_ring, t_tree

comm_topology_compare(msg_gb=140, bw_gbps=200, n=32, fanout=4)
# ring full-link ≈ 14.6s; tree relay-discounted ≈ 44.8s (no SHARP)
# → conclusion: without in-network reduction, ring wins; tree's logN depth pays off only with SHARP

6. Checklist

1. Default to ring for gradient sync — don't switch; it has the highest bandwidth utilization;
2. At large N (≥16 nodes) with SHARP-capable switches, measure tree vs ring busbw before deciding;
3. For comm-heavy workloads (All-to-All etc.), align card indices to rails and minimize cross-rail traffic;
4. Compare measured NCCL busbw with the three models; deviation >20% — check topology locality and adaptive routing (see IB Three-Plane);
5. Run the Acceptance & Baselines after each topology change and archive before/after.

---

Derived and organized by this site; method framework references the open-source book Understanding AI Infra (Apache-2.0). Version numbers and measurements are subject to on-site conditions.