A compiler uses several intermediate representations between source code and machine code. This module covers five of them: FX, Core ATen, HLO, MLIR and KTIR. For each one, you will see what it makes explicit and what it no longer records, and why different transformations need different levels.

Five compiler passes matter most for dataflow hardware: decomposition, operator fusion, layout, tiling for on-chip memory, and scheduling. This module explains what each pass decides and how a decision in one pass limits the others. It also introduces tile-level kernel languages such as Triton.

XLA is a compiler built for TPU, and its design follows from the hardware. A large systolic matrix unit is efficient only when it receives regular, correctly shaped data on every cycle. This module covers why TPU needs a whole-graph compiler, what the HLO representation makes possible, the two ways a PyTorch model reaches XLA, and the cost of requiring static shapes.

On Spyre, many decisions that a GPU leaves to caches or runtime scheduling are made by the compiler before execution. This module studies torch-spyre, the open-source frontend for Spyre, and the plans it produces. It distinguishes features that are published, features that are implemented today and features that are planned.

One attention block is compiled with XLA for TPU and with torch-spyre for Spyre. You will see where the two pipelines make the same choices and where they differ. Some transformations follow from the graph alone. Others depend on local memory size, matrix-array shape, the interconnect or the scheduling model.

This lab compiles a transformer block with torch.compile in Google Colab and measures the speedup over eager execution, weighing one-time compilation latency against the per-call steady-state saving. Students ablate the eager, aot_eager, and inductor backends to locate where the speedup originates, then detect and repair graph breaks that prevent whole-graph compilation. Reading the generated Triton and cuBLAS kernels connects operator fusion to dataflow execution. The lab adds the Week 3 row to the Accelerator Landscape Map.

This quiz covers all six modules of the course: the PyTorch compilation frontend, intermediate representations from FX to KTIR, the main compiler passes, XLA on TPU, torch-spyre on Spyre, and the two compilers side by side. It has 18 questions, including calculations of fusion traffic, padding waste and scratchpad slice size. Allow about 45 minutes and keep a calculator nearby. You need 80% to pass, and you can take the quiz up to 3 times. Your highest score counts.