09. AI Compiler

Chapter focus: This chapter focuses on compiler techniques that turn model graphs into high-performance kernels.

You Will Learn

• Triton kernel optimization patterns
• heterogeneous toolchain setup with oneAPI
• strategy differences between training and inference compilers

🧭 Reading Map

• Prerequisites: 08. Heterogeneous Computing — compilers generate code for heterogeneous hardware; understand the abstraction first
• Role: opening of Part 3, L3 — MLIR/Triton/compilation pipelines, the "model graph → high-performance kernels" middle layer
• Order: why compilers (Triton path) → MLIR architecture → PyTorch compile stack → hands-on Triton
• Next: 10. Frameworks — how the framework schedules what compilers produce

Articles

Practical Guide: Getting Started with OpenAI Triton

A deep dive into OpenAI Triton compiler, from environment setup to writing your first high-performance Vector Add kernel.

阅读文章 →

MLIR Architecture: The Unified Intermediate Representation of AI Compilers

Reproducible compile commands and IR samples dissecting MLIR: dialect layering, operations and SSA, the IR lowering pipeline, and its relationship to Triton/XLA, with engineering takeaways.

阅读文章 →

Intel oneAPI Installation Guide

A comprehensive guide for installing Intel oneAPI Base & HPC Toolkits, configuring environment variables, and running MPI tests.

阅读文章 →

PyTorch Compile Stack: Operator Fusion from TorchDynamo to TorchInductor

Reproducible scripts dissecting the PyTorch compile stack: TorchScript/TorchDynamo/TorchInductor positioning, torch.compile measured gains, fusion mechanics, IR tracing, and production mode selection, with engineering takeaways.

阅读文章 →

🚀 Deep Dive into OpenAI Triton: Building FlashAttention & High-Performance CFD from Scratch

Author: Technical Learner (Root User) Environment: Python 3.12 + NVIDIA GPU (A100/H100 Level)

阅读文章 →

CUDA Compilation Pipeline Deep Dive: nvcc → PTX → SASS

two-stage compilation, gencode combos, artifact analysis.

阅读文章 →

How AI Compilers Work: Fusion, Kernels, Autotuning

Three-layer structure, fusion types & gains, three autotuning routes, and the torch.compile stack.

Read →

Triton Kernel Practice: From Vector Addition to the FlashAttention Core

Reproducible scripts dissecting hand-written Triton kernels: vector addition, tiled GEMM, softmax and the FlashAttention core, with performance comparisons and tuning (block size/loop unrolling), and engineering takeaways.

阅读文章 →