Back to Mathematics for Machine Learning: PCA

4.0

stars

2,285 ratings

•

574 reviews

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

JS

Jul 16, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

NS

Jun 18, 2020

Relatively tougher than previous two courses in the specialization. I'd suggest giving more time and being patient in pursuit of completing this course and understanding the concepts involved.

Filter by:

By Kareem T M

•May 18, 2020

Worst Course I have ever token on Coursera, the instructor hadn't mention any examples or simplify the information.

By HARSHIT J

•Jun 11, 2020

Very tough course, the first 3 weeks are good, but the last week is as poorly explained as one can imagine

By Michael B

•May 16, 2020

If I could give it negative stars I would.

By Mohamed S

•Jun 1, 2020

topics are poorly explained and confusing

By Anton K

•Nov 14, 2020

By far, this is the best out of 3 courses in this specialization. It is hard though and in the weeks 3 and 4 I had to pause and rewind almost every 10 seconds of the videos and search some error in code labs on the web. But in the end this course showed me in great detail the process of PCA and I also learned a bit of linear algebra alongside it. Considering problems with this course, there were some points that got me a little bit dissapointed. I still don't get it why are we using the biased version of variance, sometimes the notation changed a little bit, (which is not a big problem but introduces some inconvience if the material is completely new to the learner), some of the math concepts were not covered in the "linear algebra" course. But the worst problem was a technical one: some parts of the labs that are not necessary for grading but are very important for learning were throwing errors. I hope that in the future versions they will be resolved.

By Marco v Z

•Jul 19, 2020

I was somewhat put off by critical comments about the third course in this series, but have to disagree with the reviewers. Yes, it is tougher and, yes, the instructor doesn't have the "schwung" of the other two instructors, but that doesn't affect the quality of this course. His walkthrough of the derivation of PCA is thorough and systematic, and builds on material that has been presented in the earlier lectures.

In fact, looking back on the entire specialisation, I would retrospectively grade the first two courses a notch lower (even if they're excellent), simply because they "sailed through" a bit too easily. The exercises in those courses required little thinking apart from recalling what was said in the lectures. In this course, exercises tended to go beyond or ahead of the material presented in the lectures. Solving them required active thinking, reading, and problem solving, which in the end brings a more thorough understanding.

By Heinz D

•Nov 21, 2020

Good and motivating lecturer with decent language, thank you! Challenging course but the relief at the end is great. I'd prefer if the lecturer wouldn't write his texts to the very border of the board because if I'm taking screenshots in PiP mode, the window's controls (close window, play, return to normal video mode) are overlapping.

Week 1: Pre-course survey contains the questions of rather a post-course survey. The lab / programming assignment contains misleading code segments and incomplete explanations.

Week 2: Quiz 'General inner products', dealing with 3-dimensional inner products is very challenging as the lecture only went - in an extreme hurry - through 2-dimensional examples.

Week 3: Programming Assignment contains misleading code segments / comments (e.g. contradiction concerning return variable in project_1d()).

Week 4: Video 'Problem setting and PCA objective' -> Download Link to the PCA book chapter goes to Nirvana.

By Fredrick A

•Feb 20, 2020

The coverage of PCA provided by the instructor was wide and provided me with an intuitive basis for executing the PCA algorithm in the wild. Ultimately, the subject and its various steps were easy to understand. That said, I gained many great insights watching Khan Academy videos especially ones on eigenvalues/eigenvectors. By far the hardest part of the class was implementing and executing the python code. There the devil was in, and sometimes, outside of the details. I cursed the name of the Instructor more than once (a lot more). But, in the end, because of the real life, no safety net experience, I was able to jump right into PCA (and other feature engineering projects) adding value to my team at work on day 1.

By Abdu M

•Jan 20, 2019

Best course out of the series so far. A fine balance between theory and derivations, and practice with the programming assignments. It seems that they have solved their programming assignment issues (the first one still has some problems with the grader I believe). This course does require you to have some prior experience, though, so if you are new to programming or linear algebra (not just the concepts but how to apply them) it's bets to take the first two courses with some additional help (maybe Khan academy or even MIT OCW. I will certainly refer to this course in the future, as well as the professor's book on Mathematics for ML.

By Anamitra S

•Sep 4, 2020

Even though one might read quite a few negative reviews about this course, I having completed this course certainly can tell that I learnt the most while doing this course. The course was indeed hard and challenging but the good thing that came out of this course was it gave me the ability to learn to study quite a few topics extensively on my own. The course had the book on "Mathematics for Machine Learning" which acted as a great supplement to this course. Overall, I'd ask anyone who is seriously interested in learning the extensive Math behind Machine learning, to take this course.

By Laszlo C

•Dec 6, 2019

This is an excellent course first covers statistics, looks back to inner products and projections, thereafter it connects all of that and introduces PCA. The knowledge that you've gathered throughout the first two courses gets applied here. Granted, it's more abstract and challenging than the others, I wouldn't give a worse rating just because of that. You'll need to dive into certain topics on your own and if you strengthen your coding skills for the programming exercises. Nevertheless, it's just as highly rewarding as the first two.

By Mohit J

•Dec 28, 2020

This course is too good ,difficult level of this course from other too of this specialization is more.

Having patience and more practice lead to more successful .

If anyone want to learn Machine learning course then after doing this Machine Learning course is simple because most of the thing you have learn through this course

This specialization makes you better and better and you learn many more new and interesting thing related to real world example with practice assignment

Thanks a lot for this to all the mentors

By Douglas W

•May 22, 2020

This was the most challenging of the three classes in the series. I thought the instructor did an excellent job of moving from theory to practice, and in the end I came away with a good understanding of the topic. As a developer, part of my personal learning style is to shadow these types of lectures in code. I did (or attempted) naive implementations on most slides - that definitely helped my comprehension of this challenging material. Be prepared to work hard, occasionally head scratch and you'll do fine.

By Muhammad Y A

•Nov 5, 2020

Big thanks to the teacher, this is the most challenging course among the other courses on this specialization. It took me a full 24 hours to complete the final assignment, PCA Algorithm. But, it's worth it, I really enjoyed this course besides how hard it is lmao. One more, unfortunately, there will not be much discussion on the forums, since there's a few people enroll in this course compared to others and the assignment, especially the last one was very hard, anyway hope you enjoy this course, see ya.

By Jitesh J T

•Dec 23, 2019

Hi,

The course tries to cover most of the important mathematical concepts in Mathematics applied to PCA. The assignments were a bit tough, but i guess that the road ahead when we do programming for data sets in real world applications would not be that easy. Loved the way the lectures were delivered and the programming assignments help to build a strong base for applications of linear algebra that we have done earlier.

Thanks and Regards

Jitesh Tripathi, PhD in Applied Mathematics

By Taranpreet s

•Sep 26, 2020

Although there are glitches with the submission of assignments and some of the videos by instructor are brief, I will still rate it as a 5 star for the content covered. This is 3rd course of the specialization and need solid understanding of the concepts covered in first 2. Considering the more challenging content covered in this course, Instructor did a great job. All instructors in the specialization are awesome, Would love to do more advanced courses from the same team.

By Tarek L

•Sep 11, 2019

This is a difficult course, but it really gave me an appreciation of the mathematics behind machine learning. I encourage anyone doing this course to read Deisenroth's free book Mathematics for Machine Learning (mml-book.com) to better understand the notation and technique used to get to the proofs. If anything, the rigor of this course inspired me to further pursue learning in mathematics to strengthen my machine learning foundation.

By ChristopherKing

•Apr 18, 2018

The whole content of this course is fantastic, not all details were covered in the video, but main ideas were expressed in a great way buy math formulations. Pay attention to those vectors and matrices, especially their dimensions, this will help you solve problem quickly. More important, matrix is just a way to express a bunch of similar things, knowing the meaning of those basis vectors is important.

By Sriram R

•Jun 18, 2019

This is one of toughest course in this specialization. Having said that, it was interesting to learn about the inner working of the PCA and is well taught. At times it was tough to follow and could have been better if there are some additional examples explained to reinforce the concept. Also week 4 is kind of rushed with little or no time to fully appreciate the beauty of PCA.

By Yuanfang

•Sep 7, 2019

A little more challenging than the other 2 courses in this series. The programming examples on K nearest neighbors, eigenvector fitting of facial data, and the PCA implementation are neat and rewarding. Can't help but feel there's still a great deal of math details that is only briefly mentioned - oh well there's always the free textbook to reference. Overall highly recommended.

By Sertan A

•Dec 14, 2020

Actually I was not encouraged while I am taking the course since the quest for understanding such abstract concepts required me to spend a lot of outside research and reading. Course also requires a strong understanding of Python and Anaconda (for debugging purposes). I can not say that I understand everything regarding PCA, but it became a nice foundation to built upon.

By Marcelo R

•Jul 26, 2020

Unlike the other two modules, the course is quite challenging, some details are omitted in the explanation and one has to look for them in the forum or on the internet. Some notebooks for programming have problems and need to be downloaded and run virtually. Still, the content is exciting, thanks to the Imperial College London for the course and the opportunity.

By Renato

•May 3, 2020

This course is challenging, it requires a lot of participation in the forum plus an overlook on the internet to help you out understand a little more how the vector (eigenvectors) relate to the efficiency of PCA. It is pretty interesting to understand the algorithm itself and how it works. Be aware to review a lot and take your time to understand things.

By Gergo G

•May 15, 2019

This course is really challenging. A strong mathematical background is necessary or it needs to be developed during the lectures and self-study. The professor's explanations are clear, and still lead to complex ideas which is great. Programming assignments are also difficult, however they serve as a superb opportunity to develop your skills in Python.

By Anastasios P

•Dec 26, 2019

Challenging course, a lot harder than the two previous in the specialisation. Having said that, I really enjoyed it for the insights that it gave and for actually making me learn some Python as well. With this course you need to go search and fin the necessary functions and usage to complete the assignments. The best course in the series I believe.

- Finding Purpose & Meaning in Life
- Understanding Medical Research
- Japanese for Beginners
- Introduction to Cloud Computing
- Foundations of Mindfulness
- Fundamentals of Finance
- Machine Learning
- Machine Learning Using Sas Viya
- The Science of Well Being
- Covid-19 Contact Tracing
- AI for Everyone
- Financial Markets
- Introduction to Psychology
- Getting Started with AWS
- International Marketing
- C++
- Predictive Analytics & Data Mining
- UCSD Learning How to Learn
- Michigan Programming for Everybody
- JHU R Programming
- Google CBRS CPI Training

- Natural Language Processing (NLP)
- AI for Medicine
- Good with Words: Writing & Editing
- Infections Disease Modeling
- The Pronounciation of American English
- Software Testing Automation
- Deep Learning
- Python for Everybody
- Data Science
- Business Foundations
- Excel Skills for Business
- Data Science with Python
- Finance for Everyone
- Communication Skills for Engineers
- Sales Training
- Career Brand Management
- Wharton Business Analytics
- Penn Positive Psychology
- Washington Machine Learning
- CalArts Graphic Design

- Professional Certificates
- MasterTrack Certificates
- Google IT Support
- IBM Data Science
- Google Cloud Data Engineering
- IBM Applied AI
- Google Cloud Architecture
- IBM Cybersecurity Analyst
- Google IT Automation with Python
- IBM z/OS Mainframe Practitioner
- UCI Applied Project Management
- Instructional Design Certificate
- Construction Engineering and Management Certificate
- Big Data Certificate
- Machine Learning for Analytics Certificate
- Innovation Management & Entrepreneurship Certificate
- Sustainabaility and Development Certificate
- Social Work Certificate
- AI and Machine Learning Certificate
- Spatial Data Analysis and Visualization Certificate

- Computer Science Degrees
- Business Degrees
- Public Health Degrees
- Data Science Degrees
- Bachelor's Degrees
- Bachelor of Computer Science
- MS Electrical Engineering
- Bachelor Completion Degree
- MS Management
- MS Computer Science
- MPH
- Accounting Master's Degree
- MCIT
- MBA Online
- Master of Applied Data Science
- Global MBA
- Master's of Innovation & Entrepreneurship
- MCS Data Science
- Master's in Computer Science
- Master's in Public Health