About this course: This course will introduce you to the multiple forms of parallelism found in modern Intel architecture processors and teach you the programming frameworks for handling this parallelism in applications. You will get access to a cluster of modern manycore processors (Intel Xeon Phi architecture) for experiments with graded programming exercises. This course can apply to various HPC and datacenter workloads and framework including artificial intelligence (AI). You will learn how to handle data parallelism with vector instructions, task parallelism in shared memory with threads, parallelism in distributed memory with message passing, and memory architecture parallelism with optimized data containers. This knowledge will help you to accelerate computational applications by orders of magnitude, all the while keeping your code portable and future-proof. Prerequisite: programming in C/C++ or Fortran in the Linux environment and Linux shell proficiency (navigation, file copying, editing files in text-based editors, compilation).
Who is this class for: For developers of machine learning libraries and frameworks. For innovators looking to combine machine learning and traditional computing in artificial intelligence systems. For engineers, students and researchers in computational disciplines interested in parallel computing.
Taught by: Andrey Vladimirov, Head of High-Performance Computing Research
Colfax International
| Level | Intermediate |
| Language | English |
| Hardware Req | A computer running a recent Intel Corporation Microprocessor |
| How To Pass | Pass all graded assignments to complete the course. |
| User Ratings |
- Reading: Modern Code
- Video: 1.0 Introduction
- Video: 1.1 Why this course?
- Video: 1.2 How Computers Get Faster
- Video: 1.3 Intel Architecture
- Video: 1.4 Modern Code
- Video: 1.5 What You Are Going To Learn
- Video: 1.6 Remote Access
- LTI Item: Register for cluster access
- Reading: Vectorization
- Video: 2.1 Vector Operations
- Video: 2.2 Vectorizing Your Code
- Video: 2.3.1 Automatic Vectorization
- Video: 2.3.2 Will This Vectorize?
- Video: 2.4 Guided Automatic Vectorization
- Video: 2.8.1 Stencil Introduction
- Video: 2.8 Stencil
- Reading: Code Download
- Video: 2.5 SIMD-Enabled Functions
- Video: 2.6 Vector Dependence
- Video: 2.7 Strip Mining
- Video: Numerical Integration Introduction
- Video: 2.9 Integral Vectorization
- Reading: Code Download
- Video: 2.10 Learn More
- Reading: Multithreading with OpenMP
- Video: 3.1 Cores and Threads
- Video: Demo: Forks
- Reading: Code Download
- Video: 3.2 Creating Threads
- Video: 3.3 Variable Sharing
- Video: 3.4 Parallel Loops
- Video: 3.5 Data Races Mutexes
- Video: 3.7 Parallel Reduction
- Video: Stencil Introduction
- Video: Stencil Demonstration
- Reading: Code Download
- Video: Learn More
- Reading: Memory Traffic
- Video: 4.1 Cheap Flops
- Video: 4.2 Memory Hierarchy
- Video: 4.3 High Bandwidth Memory
- Video: 4.4 Memory Allocation
- Video: Stencil Introduction
- Video: Demo: Stencil with numactl
- Video: Demo: Stencil with Memkind
- Video: 4.5 Bypassing Caches
- Video: Stencil Demonstration-Nontemporal
- Video: Stencil Demonstration-Char
- Reading: Code Download
- Video: 4.6 Locality in Space
- Video: 4.7 Locality in Time
- Video: Integral Introduction
- Video: Integral Demonstration
- Reading: Code Download
- Reading: Clusters and MPI
- Video: 5.1 Computing Clusters
- Video: 5.2 Message Passing Interface
- Video: 5.3 Programming with MPI
- Video: 5.4 Compiling and Running with MPI
- Video: 5.5 Peer-to-Peer Messaging
- Video: 5.6 Collective Communication
- Video: Stencil Introduction
- Video: Stencil Demonstration-MPI
- Reading: Code Download
- Video: Integral Introduction
- Video: Integral Demonstration
- Reading: Code Download
- Video: Learn More
Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.
Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.
Earn official recognition for your work, and share your success with friends, colleagues, and employers.
| Purchase Course | |
|---|---|
| Access to course materials | Available |
| Access to graded materials | Available |
| Receive a final grade | Available |
| Earn a shareable Course Certificate | Available |
Very well explained, very simple, well structured and interesting examples. Although, more diversity of the examples would be interesting, for example, mathematical operations (as matrix-matrix multiplication), bioinformatics, financial, and so on. Also, would be great more exercises, labs and resources.
A very good course with excellent material to enhance your knowledge. The quality of some videos could be improved (sudden changes/gaps).
Highly informative course with a steep learning curve but fruitful.
very nice course, good lecturer! I really enjoyed it!