Back to Mathematics for Machine Learning: PCA

4.0

stars

1,328 ratings

•

290 reviews

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

Jul 17, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

May 01, 2018

This course was definitely a bit more complex, not so much in assignments but in the core concepts handled, than the others in the specialisation. Overall, it was fun to do this course!

Filter by:

By Andrea V

•Jun 22, 2019

This course is hard, and contains a lot of mathematical derivations and concepts that might be overwhelming for somebody not completely fresh in maths. Nevertheless, it offers a good balance between rigour and practical application, and if some lectures turn out to be too complicated, there's always the chance to deepen the matter more quitely using the course material or online resources. I think that the course would have benefited from a more aneddoctical approach at times: for instance restating in english what the general purpose of PCA is, could help the less mathematically inclined to better seize the idea. But I know this is not always easy to do.

By Ruarob T

•Jun 30, 2019

Make sure you have time and be ready for python code debug. If you are just an average programmer with limited python exposure like me. It will take you a day to complete the programming assignment.

Note: the assignment and class VDO seems a distant - google a lot during the assignment/quiz

Note: Programming has little clue - personally, I think I spend so much time on programming (distracting me away from going back to Math review)

By Shilin G

•Jun 27, 2019

Not as good as previous two courses. I understand it is an intermediate course, but still, the video does not help you do the quiz, e.g. the video uses 2x2 matrices for example while quiz is mainly about 3x3 - then why not include a 3x3 example? Programming assignment is not clear either, some places you have to change the shape of matrix but it is not explained why this is necessary (and actually it is not). A lot of room for improvement here.

By Ustinov A

•May 28, 2019

Unfortunately, mistakes in grader and a bad python environment spoilt the impression. I lose hours because of it during 1, 2 and 4 week. It's not enough exercises last week. You should add more examples for every step of PCA for better understanding.

By Yiqing W

•Mar 28, 2019

The teaching is good but some programming assignment is not so good

By Narongdej S

•Jun 29, 2019

Confusing for beginners; the explanations are too abrupt

By Yana K

•Apr 18, 2019

Not really well structured. Too much in-depth details, too little intuition given. Didn't help to understand PCA. Had to constantly look for other resources online. Pity, because first 2 courses in the specialisation were really good.

By Patrick F

•Feb 01, 2019

The programming tasks are very bad documented and have errors.

By Andrei

•Nov 01, 2018

terrible assignments

By Rachel S

•Jul 09, 2019

After the first two courses in the specialisation, this one was truly disappointing. You are warned at the beginning that this course is challenging. This is true, but there is absolutely no reason why it should be THIS challenging. There are several factors that make this course more difficult than it needs to be. The poor pacing leads to a bizarre mix of repetitive trivial questions and vague assignments with poor explanation and over-reliance on reading external sources. Nobody wants constant hand-holding but the lack of direction will lead to you wasting far too much time chasing down minor technical errors and figuring out what on earth is being asked of you. Finishing this course was a slog and I just wanted to wash my hands of it. The first two courses in this specialisation are great and I highly recommend them, but I would not be happy if I had paid £38 for this course.

By Naveen K

•Aug 09, 2018

I've finished all the two previous courses in this specialization.I was shocked at seeing the content and programming assignments given to us.It was totally different.They expect a lot from us.Content is not up to the mark.First two courses was awesome.But this course is an exact opposite to the first two.Totally disappointed!! I was hoping to finish this specialization.But it seems I cannot. I didn't expect this.

By Ong J R

•Aug 11, 2018

Concepts weren't taught well and programming exercises are full of errors. Very difficult to debug and find out if I am on track during the programming exercises. Lecturer lacks passion and ability to convey core concepts well to audience. Hard to follow up on the mathematical derivation with the simple stuff that we were taught in module 1 and 2.

By Valeria B

•Jun 26, 2019

Too few examples given during the lessons. More examples could greatly improve understanding and the solution of quizzes and programming assignment.

I had to integrate this course with multiple sources I looked up for by myself, so I'm really wondering if I wisely spent my money on this course.

By 용석 권

•Jan 30, 2019

Programming assignments' quality is too bad to follow it. Their lecture's explanation and assignments' notation are not matched. Moreover, the code is sometimes ridiculous.

By Fredrick A

•Feb 21, 2020

The coverage of PCA provided by the instructor was wide and provided me with an intuitive basis for executing the PCA algorithm in the wild. Ultimately, the subject and its various steps were easy to understand. That said, I gained many great insights watching Khan Academy videos especially ones on eigenvalues/eigenvectors. By far the hardest part of the class was implementing and executing the python code. There the devil was in, and sometimes, outside of the details. I cursed the name of the Instructor more than once (a lot more). But, in the end, because of the real life, no safety net experience, I was able to jump right into PCA (and other feature engineering projects) adding value to my team at work on day 1.

By Abdu M

•Jan 20, 2019

Best course out of the series so far. A fine balance between theory and derivations, and practice with the programming assignments. It seems that they have solved their programming assignment issues (the first one still has some problems with the grader I believe). This course does require you to have some prior experience, though, so if you are new to programming or linear algebra (not just the concepts but how to apply them) it's bets to take the first two courses with some additional help (maybe Khan academy or even MIT OCW. I will certainly refer to this course in the future, as well as the professor's book on Mathematics for ML.

By Laszlo C

•Dec 06, 2019

This is an excellent course first covers statistics, looks back to inner products and projections, thereafter it connects all of that and introduces PCA. The knowledge that you've gathered throughout the first two courses gets applied here. Granted, it's more abstract and challenging than the others, I wouldn't give a worse rating just because of that. You'll need to dive into certain topics on your own and if you strengthen your coding skills for the programming exercises. Nevertheless, it's just as highly rewarding as the first two.

By Jitesh J T

•Dec 24, 2019

Hi,

The course tries to cover most of the important mathematical concepts in Mathematics applied to PCA. The assignments were a bit tough, but i guess that the road ahead when we do programming for data sets in real world applications would not be that easy. Loved the way the lectures were delivered and the programming assignments help to build a strong base for applications of linear algebra that we have done earlier.

Thanks and Regards

Jitesh Tripathi, PhD in Applied Mathematics

By Tarek L

•Sep 11, 2019

This is a difficult course, but it really gave me an appreciation of the mathematics behind machine learning. I encourage anyone doing this course to read Deisenroth's free book Mathematics for Machine Learning (mml-book.com) to better understand the notation and technique used to get to the proofs. If anything, the rigor of this course inspired me to further pursue learning in mathematics to strengthen my machine learning foundation.

By ChristopherKing

•Apr 18, 2018

The whole content of this course is fantastic, not all details were covered in the video, but main ideas were expressed in a great way buy math formulations. Pay attention to those vectors and matrices, especially their dimensions, this will help you solve problem quickly. More important, matrix is just a way to express a bunch of similar things, knowing the meaning of those basis vectors is important.

By Sriram R

•Jun 18, 2019

This is one of toughest course in this specialization. Having said that, it was interesting to learn about the inner working of the PCA and is well taught. At times it was tough to follow and could have been better if there are some additional examples explained to reinforce the concept. Also week 4 is kind of rushed with little or no time to fully appreciate the beauty of PCA.

By Yuanfang

•Sep 07, 2019

A little more challenging than the other 2 courses in this series. The programming examples on K nearest neighbors, eigenvector fitting of facial data, and the PCA implementation are neat and rewarding. Can't help but feel there's still a great deal of math details that is only briefly mentioned - oh well there's always the free textbook to reference. Overall highly recommended.

By Gergo G

•May 15, 2019

This course is really challenging. A strong mathematical background is necessary or it needs to be developed during the lectures and self-study. The professor's explanations are clear, and still lead to complex ideas which is great. Programming assignments are also difficult, however they serve as a superb opportunity to develop your skills in Python.

By Anastasios P

•Dec 26, 2019

Challenging course, a lot harder than the two previous in the specialisation. Having said that, I really enjoyed it for the insights that it gave and for actually making me learn some Python as well. With this course you need to go search and fin the necessary functions and usage to complete the assignments. The best course in the series I believe.

By Idris R

•Nov 02, 2019

Great, challenging course. The instructor will expect much of you as the material is not spoon fed. At times this is frustrating but yet that's the best way to build your own intuition. This is a *hard* course and I imagine most of machine learning is like this. Fun, rewarding, and challenging. You'll flex your math and programming muscles.