This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.

This course is part of the Mathematics for Machine Learning Specialization

# Mathematics for Machine Learning: PCA

Offered By

## About this Course

### Learner Career Outcomes

## 50%

## 48%

### Skills you will gain

### Learner Career Outcomes

## 50%

## 48%

#### 100% online

#### Course 3 of 3 in the

#### Flexible deadlines

#### Intermediate Level

#### Approx. 18 hours to complete

#### English

## Syllabus - What you will learn from this course

**5 hours to complete**

## Statistics of Datasets

Principal Component Analysis (PCA) is one of the most important dimensionality reduction algorithms in machine learning. In this course, we lay the mathematical foundations to derive and understand PCA from a geometric point of view. In this module, we learn how to summarize datasets (e.g., images) using basic statistics, such as the mean and the variance. We also look at properties of the mean and the variance when we shift or scale the original data set. We will provide mathematical intuition as well as the skills to derive the results. We will also implement our results in code (jupyter notebooks), which will allow us to practice our mathematical understand to compute averages of image data sets.

**5 hours to complete**

**8 videos**

**6 readings**

**3 practice exercises**

**4 hours to complete**

## Inner Products

Data can be interpreted as vectors. Vectors allow us to talk about geometric concepts, such as lengths, distances and angles to characterise similarity between vectors. This will become important later in the course when we discuss PCA. In this module, we will introduce and practice the concept of an inner product. Inner products allow us to talk about geometric concepts in vector spaces. More specifically, we will start with the dot product (which we may still know from school) as a special case of an inner product, and then move toward a more general concept of an inner product, which play an integral part in some areas of machine learning, such as kernel machines (this includes support vector machines and Gaussian processes). We have a lot of exercises in this module to practice and understand the concept of inner products.

**4 hours to complete**

**8 videos**

**1 reading**

**4 practice exercises**

**4 hours to complete**

## Orthogonal Projections

In this module, we will look at orthogonal projections of vectors, which live in a high-dimensional vector space, onto lower-dimensional subspaces. This will play an important role in the next module when we derive PCA. We will start off with a geometric motivation of what an orthogonal projection is and work our way through the corresponding derivation. We will end up with a single equation that allows us to project any vector onto a lower-dimensional subspace. However, we will also understand how this equation came about. As in the other modules, we will have both pen-and-paper practice and a small programming example with a jupyter notebook.

**4 hours to complete**

**6 videos**

**1 reading**

**2 practice exercises**

**5 hours to complete**

## Principal Component Analysis

We can think of dimensionality reduction as a way of compressing data with some loss, similar to jpg or mp3. Principal Component Analysis (PCA) is one of the most fundamental dimensionality reduction techniques that are used in machine learning. In this module, we use the results from the first three modules of this course and derive PCA from a geometric point of view. Within this course, this module is the most challenging one, and we will go through an explicit derivation of PCA plus some coding exercises that will make us a proficient user of PCA.

**5 hours to complete**

**10 videos**

**5 readings**

**1 practice exercise**

### Reviews

#### 4.0

##### TOP REVIEWS FROM MATHEMATICS FOR MACHINE LEARNING: PCA

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

Great capstone for the three-class Mathematics for Machine Learning series. Assignments were way harder and programming debugging skills had to be appropiate in order to finish the class.

This course was definitely a bit more complex, not so much in assignments but in the core concepts handled, than the others in the specialisation. Overall, it was fun to do this course!

This course is well worth the time. I have a better understanding of one of the most foundational and biologically plausible machine learning algorithms used today! Love it.

Programming assignment for week 1 wastes to much time due to lack of instructions.\n\nThe notebook also does not work...(maybe locally , but I have other things to do).

Course content tackles a difficult topic well. Only flaw is that programming assignments are poorly designed in some places and are quite difficult to pick up at times.

This course demystifies the Principal Components Analysis through practical implementation. It gives me solid foundations for learning further data science techniques.

Teaching pacing is good, and clear in explanation. It will be good if there are some examples about how we should apply all these theories to some real problems.

### About Imperial College London

## About the Mathematics for Machine Learning Specialization

## Frequently Asked Questions

When will I have access to the lectures and assignments?

Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.

What will I get if I subscribe to this Specialization?

When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.

What is the refund policy?

Is financial aid available?

More questions? Visit the Learner Help Center.