CUDA Compilation Pipeline Deep Dive: The Optimization Space from nvcc to SASS

Experiment Goal

"Why does the same CUDA code perform so differently across architectures?" "How should -arch and -code be filled in?" "Should optimization happen at the source level or by reading kernel assembly?" — these questions all point to a pipeline many skip: the CUDA compilation pipeline.

• Understand nvcc's layered artifacts (C++ source → PTX → SASS/cubin) and when each is produced;
• Master correct -arch/-code/-gencode usage to avoid "code runs but misses new-hardware features";
• Learn to inspect actual generated SASS with cuobjdump/nvdisasm, building source-to-machine-code optimization judgment.

This article is a pipeline-analysis route; commands and artifact analysis are generic practice without fabricating performance benchmarks.

Stop 1: Understand nvcc's Two-Stage Compilation

CUDA compilation is two-stage, fundamentally different from ordinary C++ compilation:

kernel.cu
   │  nvcc (host compiler + device compiler cooperate)
   ▼
PTX (Parallel Thread Execution ISA — architecture-agnostic IR)
   │  ptxas (inside nvcc or standalone)
   ▼
SASS (architecture-specific machine code, e.g. sm_80/sm_90 ISA)
   │  cubin or embedded into host executable
   ▼
Runtime: CUDA driver can JIT from PTX, or load SASS directly

Key facts:

• PTX is portable: one PTX can theoretically JIT to newer architectures (forward-compatible), at the cost of JIT overhead and suboptimal tuning;
• SASS is locked: only targets the -arch at compile time; changing architecture requires recompilation;
• Host and device compile separately: -arch affects the device side; host-side optimization is gcc/clang's job.

Stop 2: Correct Use of -arch / -code / -gencode

This is the most common error zone. The three parameters:

Flag	Meaning	Example
-arch=sm_XX	PTX target architecture (virtual architecture)	-arch=sm_90
-code=sm_XX	SASS target architecture (real architecture)	-code=sm_90
-gencode arch=...,code=...	Multi-architecture fat binary (repeatable)	-gencode arch=compute_90,code=sm_90

Practice rules:

# Common mistake: only -arch=sm_90 (and using arch as the real arch name)
nvcc -arch=sm_90 kernel.cu          # arch should use compute_XX or sm_XX

# Correct 1: single deployment arch, maximum optimization
nvcc -gencode arch=compute_90,code=sm_90 kernel.cu

# Correct 2: multi-arch compatibility (fatbin) for new+old cards
nvcc -gencode arch=compute_80,code=sm_80 \
     -gencode arch=compute_90,code=sm_90 \
     kernel.cu

# Correct 3: include PTX (code=compute_XX) for forward-compat headroom
nvcc -gencode arch=compute_90,code=sm_90 \
     -gencode arch=compute_90,code=compute_90 \
     kernel.cu

Engineering observation: in mixed clusters (Chapter 07), "code runs" masks a lot — if the fatbin only has sm_90 SASS, jobs scheduled to A100 (sm_80) nodes either JIT (if PTX present) or fail (no PTX, no matching SASS). Which architectures to deploy and which gencode combinations to package should align with the cluster's card inventory (Chapter 01 selection), not the build defaults.

Stop 3: Inspect Actual Artifacts — cuobjdump & nvdisasm

Don't just run after compiling; look at what's actually in the artifact:

# Inspect CUDA objects embedded in the executable/library
cuobjdump -elf kernel_exe | grep -E "arch|nvdisasm" | head
# List which architectures' cubin/PTX are in the fatbin
cuobjdump -sass kernel_exe | head -20
# Disassemble a specific kernel to SASS
nvdisasm -c kernel_exe | head -40

Three things to look at when reading SASS:

1. Register usage (R count): too many registers → high occupancy pressure → occupancy drops; too few → local variable spills to local memory. Use --ptxas-options=-v at compile time for register/spill stats.
2. Memory instruction forms: ratio of LDG/STG (global memory) vs LDS/STS (shared memory) tells whether the kernel is memory-bound; presence of LDL/STL (local) indicates spills happened.
3. Instruction mix: FFMA (fused multiply-add), TENSOR-class instructions (Tensor Core, e.g. HMMA/IMMA) — confirm the code really uses hardware features.

# Get register/spill/instruction stats at compile time (key habit)
nvcc -arch=sm_90 -Xptxas -v -o kernel kernel.cu
# Output fields: regs, spill loads/stores, used global/local/shared memory

Stop 4: Turn the Pipeline into an Optimization Tool

1. Architecture Target Determines Feature Availability

The same matmul code runs the FFMA path on sm_80 but can use WGMMA (warpgroup matrix instructions) on sm_90. Much "unoptimized code" is actually "misaligned architecture target":

# Check whether the artifact actually uses Tensor Core paths
cuobjdump -sass kernel_exe | grep -iE "HMMA|IMMA|WGMMA" | head -3
# No output → kernel may not use tensor paths (source lacks wmma/mma, or -arch too low)

2. PTX-Level vs SASS-Level Optimization

• Source level (first choice): change algorithms, memory access patterns, __restrict__, shared-memory tiling — biggest gain, maintainable;
• PTX level (middle): hand-written/inlined PTX (e.g. __syncwarp, special instructions) — only when source can't express it;
• SASS level (last resort): manual tweaks after disassembly — rare; usually for "confirming the compiler generated expected instructions" rather than editing SASS directly.

Engineering observation: 90% of optimization problems are solved at the source level; the greatest value of reading SASS is "verification" — confirming your optimization intent is reflected in machine code (vectorization, no spills, tensor paths), not rewriting assembly.

3. Compilation Strategy for Multi-Arch Deployment

• Training/inference services: pin -gencode to actual deployed cards; package only necessary architectures (smaller size, full features);
• Generic libraries/tools: multi-arch fatbin + PTX for compatibility;
• Pin in CI: hard-code -gencode combinations into build scripts and CI checks (e.g. "artifact must contain sm_90 SASS") to prevent per-team compile drift.

Artifact-Analysis Checklist

• [ ] -gencode combinations aligned with cluster card inventory (with PTX compatibility decision)
• [ ] -Xptxas -v at compile time: registers and spills checked; no unexpected spilling
• [ ] cuobjdump -sass confirms key kernels use expected instruction paths (tensor/vectorized)
• [ ] Multi-arch fatbin contains all required SASS (plus PTX when needed)
• [ ] CI/build scripts pin compile flags, consistent with delivery baseline (Chapter 01)

Engineering Takeaways

• The compilation pipeline is the "last layer of truth": when performance debates stall at the source level, artifacts give objective answers — registers, spills, instruction shapes at a glance.
• Architecture target = feature switch: new-arch features (new Tensor Core instructions, larger register files) only activate when -arch is aligned; deployment-arch vs compile-target mismatch is the main source of hidden performance loss.
• Make verification a habit: -Xptxas -v + cuobjdump is a ten-minute habit that turns optimization from "feels effective" to "confirmed effective".

Next article preview: with kernels compiling and running efficiently, we return to the cluster view — the next batch will strengthen Chapter 13 application landing with more real-world engineering closed loops, and Chapter 10 pretraining with engineering deep dives. Stay tuned.