AI-HPC Converged Architecture: The Knowledge Base as a Book

This knowledge base is organized like a professional book: 15 chapters, progressing from accounting to hardware, systems, compilers, models and applications. This "core architecture" page is the book's overview — see how layers and chapters depend on each other first, then decide where to start reading.

Above: the AI-HPC full-stack overview (click to enlarge), from infrastructure through system software, LLM technology, to applications and scientific computing.

Six Parts: 15 Chapters in Six Progressive Layers

The whole book is organized into L0 → L5 layers, each spanning one or more chapters. Layers depend on each other progressively: every design decision in a layer builds on the physical constraints of the layer below.

Part	Layer	Chapters	What this part covers	What you gain
Part 0	L0 Quantitative Methods	01. AI Infra Quant	Do the math first: memory, compute and bandwidth ledgers	Derive any architecture choice yourself
Part 1	L1 Hardware & Infrastructure	02. Hardware & Chips → 03. Cluster Architecture → 04. Network → 05. Storage	Every brick of the compute base	Read server/cluster/network/storage specs and topologies
Part 2	L2 System Software & Scheduling	06. Provisioning → 07. Scheduling & Cloud → 08. Heterogeneous	Turning hardware into a usable platform	Deliver and schedule a cluster independently
Part 3	L3 Compilation & Frameworks	09. AI Compiler → 10. Frameworks	From model graph to high-performance kernels	Understand how operators become GPU code
Part 4	L4 Model Engineering	11. Pre-training → 12. Distributed Training → 13. Inference	Efficient training and inference at scale	Design and tune training/inference plans
Part 5	L5 Applications & Science	14. Industry Apps → 15. AI×Science	Turning compute into business and scientific output	Land industry apps, understand AI×Science

Inter-Layer Dependencies: Why This Order

Chapter prerequisites are not arbitrary — they follow the flow of data and knowledge:

L0 Quantitative Analysis (runs through the whole book, revisit anytime)
        │ defines "how much data to move, how much compute needed"
        ▼
L1 Hardware Base (01→04) ─── physical constraints: bandwidth, latency, capacity, power
        │ provisioning/scheduling must understand hardware topology (05 depends on 02; 03 caps NCCL)
        ▼
L2 System Software (05→07) ─── turn hardware into a usable platform
        │ compilers/frameworks depend on heterogeneous drivers and scheduling (08 on 07; 09 on 08)
        ▼
L3 Compilation & Frameworks (08→09) ─── operator generation and framework runtime
        │ training performance depends on framework + comm libraries (11 on 09/03)
        ▼
L4 Model Engineering (10→12) ─── training and inference engineering
        │ apps depend on inference engines and scheduling (13 on 12/06)
        ▼
L5 Applications & Science (13→14) ─── business and scientific output

Three Reading Paths

Choose your entry by role — you do not need to read chapter 1 through 15:

Your goal	Recommended path	Why
Full-stack cognition	L0 → L1 → L2 (01 → 02 → … → 08)	Do the math, learn the hardware, then delivery — the fastest complete picture
Deliver clusters independently	L1 → L2 full (02 → 08; the L40S series lives in 03)	Hardware checks, network, provisioning, scheduling — the full delivery chain
Algorithm/model engineer	L4 → L5 (10 → 14), revisit L0 as needed	Focus on training and inference; return to L0 when resource-bound

Full-Stack Technology Components (vs. the Diagram)

Layer	Technology	Chapter
Applications	PyTorch, TensorFlow, OpenMM, GROMACS, vLLM	09 / 11 / 12 / 13
Scheduling	Slurm, Kubernetes, containers	06
Communication	MPI, NCCL, UCC	02 / 03
Drivers	CUDA, ROCm, domestic accelerator runtimes	07
Interconnect	NVLink, InfiniBand, RoCE v2	03
Storage	Lustre, BeeGFS, GPFS, All-Flash cache	04

Tip: first-time visitors should start at the Knowledge Overview. To place a single article in context, return to this page to see which layer it belongs to and which chapter precedes it.