Back to Mathematics for Machine Learning: PCA

4.0

stars

2,280 ratings

•

571 reviews

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

JS

Jul 16, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

NS

Jun 18, 2020

Relatively tougher than previous two courses in the specialization. I'd suggest giving more time and being patient in pursuit of completing this course and understanding the concepts involved.

Filter by:

By sairavikanth t

•Apr 29, 2018

Lot of Math. Couldn't get proper intuition regarding PCA, was lost in understanding math equations

By Jessica P

•Aug 6, 2019

I agree with the others. Course didn't merge well with the 1st two which were perfect!

By Clara M L

•May 1, 2018

Not as good as the other two courses but still very intuitive

By Tobias L

•Sep 10, 2020

PCA is derived using the mathematical approach. I liked this, it was systematically done by the lecturer without leaving me puzzled on how he did it. If you do not like maths, sums, the delta operator and so forth this might not be the right course for you.

However, the course is quite buggy and needs a mayor overhaul. Quizzes in the videos have no answers, the practial assignments have quite some bugs - outside the code, we are supposed to edit.

Fixes for these bugs can found in the forums or - given enough Python and NumPy knowledge - be fixed by one self. Yet the instructors do not fix these once and for all. To me this seems lazy and I expect more from a course that is paid for and has an audience that is mostly doing this during afterhours and wants to learn something about PCA and not on how to find workarounds to please the AutoGrader. Without these issues and I would have given the course a 5-star rating.

By Tony J

•Oct 2, 2020

This course is remarkable for the rigour it takes you through to understand the PCA. If you make it through and understand everything it is well worth it.

Unfortunately, you will almost certainly need to supplement the course with materials, videos, and theory from elsewhere, because a great bulk of the lectures are not intuitive, you might as well be learning from a rather obtuse textbook.

The assignments as many have mentioned, continue to have bugs and errors, despite the recent attentiveness of the course staff on the forums.. hopefully they will be fixed soon. At least they've finally included a Numpy tutorial.

Overall though. I have to say, this course, if you stick with it, will force you to get a robust grasp of the linear algebra that you've been taught so far, and it is a good exercise. Although, it's certainly not a smooth ride.

By Shilin G

•Jun 27, 2019

Not as good as previous two courses. I understand it is an intermediate course, but still, the video does not help you do the quiz, e.g. the video uses 2x2 matrices for example while quiz is mainly about 3x3 - then why not include a 3x3 example? Programming assignment is not clear either, some places you have to change the shape of matrix but it is not explained why this is necessary (and actually it is not). A lot of room for improvement here.

By Patrick G

•May 17, 2020

Very challenging course in terms of computing ; one have to always go to the forum which is very active and function like StackOverFlow. You must have somme skills in PYthon, an intermediate level in matrix algebra and deserve a high amount of time and effort to do the assignments but at the end you get a good comprehension of PCA algorithm.

By Ustinov A

•May 28, 2019

Unfortunately, mistakes in grader and a bad python environment spoilt the impression. I lose hours because of it during 1, 2 and 4 week. It's not enough exercises last week. You should add more examples for every step of PCA for better understanding.

By Yougui Q

•Jun 2, 2020

The course is relatively harder than the other two courses in this specialization. The lecturer didn't provide understandable examples while demonstrating the concepts. The grader for Python assignments didn't function well either.

By Yiqing W

•Mar 28, 2019

The teaching is good but some programming assignment is not so good

By Narongdej S

•Jun 29, 2019

Confusing for beginners; the explanations are too abrupt

By Jian L

•Oct 23, 2020

The topic of this course is an important and extremely useful concept and approach for Machine Learning and other applications. However, the way it was taught was ineffective for learning (to me personally and I suspect to many others as well) regardless of our different backgrounds or lack of strong math skills.

Although the concept of PCA is rather simple, its computations and the interpretations & meanings of each transformation are challenging without examples and visualizations. We did have some examples in the course, but not detailed and complete enough to fully understand PCA intuitively and thoroughly.

One ended up spending huge amount of time reading additional relevant articles on internet, fishing out useful information from Discussion Forums, yet still struggling a great deal to write codes for completing assignments. Even though the codes run through without returning mistakes and one gets "pass", there is not much satisfaction and joy one feels in learning because sometimes, the initial learning from listening to the lectures was so insufficient and inadequate for doing the assignment that one simply completes the task doing all one can without further understanding and solidification of the knowledge.

This feeling is in such a huge contrast to those when learning the two other related courses in the Specialization (Linear Algebra and Multivariate Calculus) during which one enjoyed immensely the learning process as well as the content. Here I do not think that one can attribute the difference to the seemingly higher level of difficulty of PCA. The instructors for those two courses did an excellent job to explain things intuitively and thoroughly.

Marc is an extremely knowledgeable and competent professional. By changing the teaching style more in tune with the learners, I am sure more people will learn a great deal from this course and enjoy the learning process in the meantime. Thank you for the hard work put into teaching this course all the same.

p.s. If the teaching staff involved can fix all technical issues or mistakes related to the Assignments, it will greatly benefit the learners. Thank you.

By Kenny C

•Jul 22, 2020

This course was very frustrating. I would say that I'm quite competent in math, but I still struggled, not necessarily because the content is challenging, but because the instructions are unclear. I like that the lectures go through derivations in detail, but the instructor often skips steps. Sometimes he would reference a property of matrices that were not talked about, and I would have to spend half an hour researching what that property was to follow what was happening. The quizzes were minimally helpful, as they were merely the same computation question repeated throughout the quiz, which does not help to build intuitive understanding. The programming assignments are unclear on instructions and had many bugs, even in the pre-written parts. A lot of time was spent on reading the NumPy documentation, as the assignments gave little indication of what functions should be used and how they should be used. Overall, despite having a mathematical derivation of PCA, the course is very confusing and frustrating, perhaps even to those competent in this area of study.

By Mohammad O B S

•Aug 22, 2020

Relative to the first two courses, this one unforutanately focused a lot less on building the intuition and more on proofs and theorems. The instructor did not offer insight into the "why" and "how" of projections and it was left on us to figure out how to connect eigenvectors and projections to derive PCA. The instructor also offered zero insight into the inner products properties. Big thanks to Susan Huang for explaining so many challenging and theoretical concepts on discussion forums in such beautiful detail.

By Astankov D A

•May 26, 2020

Although the lecturer admits that the course is quite challenging at times, it is a poor justification for the terrible assignments with close to zero explanations, errors in functions and lots of misfunctioning code in general where the notebook keeps spinning in an infinite loop. I was very hesitant while rating this course - sometimes I wanted to give it 4 stars and sometimes just a single one. I ended up with just two due to the really bad final programming assignment.

By Karl

•May 30, 2020

Pretty bad in comparison to the previous 2 courses. Not sure if the topic was just harder or it was presented less clearly. Assignments were confusing and I spent a lot of time trying to work out what I was supposed to be doing. More relevant practice questions might have been better. Also course felt slightly detached and maybe collaboration between the tutors which seemed to be there in the previous course should have happened here.

By Yana K

•Apr 18, 2019

Not really well structured. Too much in-depth details, too little intuition given. Didn't help to understand PCA. Had to constantly look for other resources online. Pity, because first 2 courses in the specialisation were really good.

By Ali K

•Jun 3, 2020

the instructor is knowledgeable but he has no teaching skills what so ever. He makes things very confusing. An example at the end would be very useful. No step-wise algorithm is provided.

By Patrick F

•Feb 1, 2019

The programming tasks are very bad documented and have errors.

By Andrei

•Nov 1, 2018

terrible assignments

By Steve

•Sep 5, 2020

Very sketchy presentation of complex material. Each lecture averaged around 5-6 minutes when they should have been 15-20 minutes. As a result the instructor glossed over the material without adequate explanations and derivations. And no one at Imperial College seems to be responding to recent posts in the discussion forums.

By Shubhayu D

•Jun 13, 2020

The first two courses in the specialization were extremely good. However, this course is nowhere close to them. Neither does the instructor provide enough intuition, nor do the assignments help in the learning process.

By Abhishek S

•Jun 7, 2020

The first two courses of this specialisation were awesome PCA being a hard topic is difficult to understand but the course was boring and not good compared to previous two.

By 용석 권

•Jan 29, 2019

Programming assignments' quality is too bad to follow it. Their lecture's explanation and assignments' notation are not matched. Moreover, the code is sometimes ridiculous.

By Benjamin F

•Nov 18, 2019

The didactic value of this course is rather low. The lectures do not explain the very concepts required to sovle the subsequent assigments, or do it in a very poor way.

- Finding Purpose & Meaning in Life
- Understanding Medical Research
- Japanese for Beginners
- Introduction to Cloud Computing
- Foundations of Mindfulness
- Fundamentals of Finance
- Machine Learning
- Machine Learning Using Sas Viya
- The Science of Well Being
- Covid-19 Contact Tracing
- AI for Everyone
- Financial Markets
- Introduction to Psychology
- Getting Started with AWS
- International Marketing
- C++
- Predictive Analytics & Data Mining
- UCSD Learning How to Learn
- Michigan Programming for Everybody
- JHU R Programming
- Google CBRS CPI Training

- Natural Language Processing (NLP)
- AI for Medicine
- Good with Words: Writing & Editing
- Infections Disease Modeling
- The Pronounciation of American English
- Software Testing Automation
- Deep Learning
- Python for Everybody
- Data Science
- Business Foundations
- Excel Skills for Business
- Data Science with Python
- Finance for Everyone
- Communication Skills for Engineers
- Sales Training
- Career Brand Management
- Wharton Business Analytics
- Penn Positive Psychology
- Washington Machine Learning
- CalArts Graphic Design

- Professional Certificates
- MasterTrack Certificates
- Google IT Support
- IBM Data Science
- Google Cloud Data Engineering
- IBM Applied AI
- Google Cloud Architecture
- IBM Cybersecurity Analyst
- Google IT Automation with Python
- IBM z/OS Mainframe Practitioner
- UCI Applied Project Management
- Instructional Design Certificate
- Construction Engineering and Management Certificate
- Big Data Certificate
- Machine Learning for Analytics Certificate
- Innovation Management & Entrepreneurship Certificate
- Sustainabaility and Development Certificate
- Social Work Certificate
- AI and Machine Learning Certificate
- Spatial Data Analysis and Visualization Certificate

- Computer Science Degrees
- Business Degrees
- Public Health Degrees
- Data Science Degrees
- Bachelor's Degrees
- Bachelor of Computer Science
- MS Electrical Engineering
- Bachelor Completion Degree
- MS Management
- MS Computer Science
- MPH
- Accounting Master's Degree
- MCIT
- MBA Online
- Master of Applied Data Science
- Global MBA
- Master's of Innovation & Entrepreneurship
- MCS Data Science
- Master's in Computer Science
- Master's in Public Health