This module covers how a PyTorch program becomes a graph that a compiler can process. It describes the structure of a torch.fx graph, including node types, targets, arguments, and the output node. It then identifies what the traced graph does not contain, such as concrete tensor shapes and memory layout, and explains the guards that record the assumptions under which a traced graph is valid. The module concludes with the sequence of compiler stages that take a captured graph to hardware execution.

This module places accelerator architectures in a two-dimensional design space defined by memory model and compile-time scheduling commitment. It describes the GPU warp as the unit of SIMT execution and then presents the TPU systolic array step by step, from weight loading to the completed wavefront. It covers the TPU's compiler-managed dataflow and the organization of TPU pods connected by the inter-chip interconnect. It then describes the Groq Tensor Streaming Processor, including its functional slices, superlanes, and the separate paths for instructions and tensor data. The module ends with a comparison of five architectures across six design dimensions, which forms the Week 2 row of the Accelerator Landscape Map.

Moving data between chips is slow and costs energy. Cerebras keeps most of that traffic on a single wafer-sized chip. This module looks at how that choice affects memory, routing, compilation and the handling of manufacturing defects.

Each core on IBM Spyre has a small scratchpad that software manages directly. As a result, the compiler’s choices about tile size and data lifetime decide much of the performance. This module is based on public architecture and software documentation, and it separates the software path in use today from the KTIR work that is still being developed.

The attention computation stays the same and the hardware changes. You will follow it through the Spyre and TPU compiler stacks and explain each difference in their output from the hardware, not from the names of compiler passes.

This lab traces a transformer block into a dataflow graph with torch.fx and visualizes it as a DAG in Google Colab. Students annotate every node with FLOPs and bytes, compute per-node arithmetic intensity, and classify each node as compute-bound or memory-bound against their GPU's ridge point. Re-classifying the same graph against a dataflow accelerator's scratchpad ridge shows the partition shift while the graph stays unchanged.

This quiz covers all six modules of the course: dataflow firing rules and the six-layer execution model, PyTorch FX graphs, the six-question comparison of accelerators, Cerebras, Spyre, and the attention case study on Spyre and TPU. It has 19 questions, including shape-bucket padding and SRAM capacity calculations. Allow about 50 minutes and keep a calculator nearby. You need 80% to pass, and you can take the quiz up to 3 times. Your highest score counts.