Back to Mathematics for Machine Learning: PCA

stars

2,340 ratings

•

587 reviews

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

JS

Jul 16, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

NS

Jun 18, 2020

Relatively tougher than previous two courses in the specialization. I'd suggest giving more time and being patient in pursuit of completing this course and understanding the concepts involved.

Filter by:

By SUSAN H M

•Oct 21, 2019

Some people complained about this course or specialization such that there was not enough help or the instructors were so lazy that a lot of the time, they just rushed through and ignored the details. But I LIKE this course and this specialization. All those complaints expressed by some people worked to my advantages. I do not need someone to monitor me at every step, so I do not mind the instructors stay out of discussion forum most of the time. Indeed, the most recent reply from an instructor I saw was a year old (or 2-year old, don't remember exactly). I would have genuinely hated this course or specialization if the instructors intervened at every step.

Throughout this specialization, I was able to discuss course contents and algorithm codes freely on the discussion forum. That really enhanced the learning experiences. I did not feel that my learning journey was micromanaged and I truly valued that freedom to develop my own understanding and insights.

I gave all three courses in the specialization a five-star review (would give six stars if that is available). I would definitely take another course by the instructors on Coursera.

By Kathleen D

•Dec 10, 2020

This course covers critical material, but unfortunately does not present them well. I found myself having to constantly find outside resources for clearer explanations. For many topics, the course itself makes no attempt to explain them, but just presents links to dense reading material and, in a couple of cases, Wikipedia articles. In my opinion, if you're linking to Wikipedia for core components of the course, you're no longer justified charging for that course. The lectures are spent deriving formulas, but providing little or no intuition about what they mean or how they are used. The course does not prepare one well for the final lab in particular, which felt like a disconnect from the lectures, had unclear explanations of what to do, and contained errors in the provided code. I did power through it, and I did learn something about PCA, but feel like I needed there to be something more to cement what I learned in practical understanding.

By Tobias L

•Sep 10, 2020

PCA is derived using the mathematical approach. I liked this, it was systematically done by the lecturer without leaving me puzzled on how he did it. If you do not like maths, sums, the delta operator and so forth this might not be the right course for you.

However, the course is quite buggy and needs a mayor overhaul. Quizzes in the videos have no answers, the practial assignments have quite some bugs - outside the code, we are supposed to edit.

Fixes for these bugs can found in the forums or - given enough Python and NumPy knowledge - be fixed by one self. Yet the instructors do not fix these once and for all. To me this seems lazy and I expect more from a course that is paid for and has an audience that is mostly doing this during afterhours and wants to learn something about PCA and not on how to find workarounds to please the AutoGrader. Without these issues and I would have given the course a 5-star rating.

By Tony J

•Oct 2, 2020

This course is remarkable for the rigour it takes you through to understand the PCA. If you make it through and understand everything it is well worth it.

Unfortunately, you will almost certainly need to supplement the course with materials, videos, and theory from elsewhere, because a great bulk of the lectures are not intuitive, you might as well be learning from a rather obtuse textbook.

The assignments as many have mentioned, continue to have bugs and errors, despite the recent attentiveness of the course staff on the forums.. hopefully they will be fixed soon. At least they've finally included a Numpy tutorial.

Overall though. I have to say, this course, if you stick with it, will force you to get a robust grasp of the linear algebra that you've been taught so far, and it is a good exercise. Although, it's certainly not a smooth ride.

By Sergii T

•Dec 22, 2018

Course is targeted more on pure math derivations, rather then real world applications. For my opinion, it doesn't fit well with other courses in this specialisation. it goes too Deep in math derivations. It should fit for students interested in mathematics and not engineers, who want to get more insights in ML related math.

By Jian L

•Oct 23, 2020

The topic of this course is an important and extremely useful concept and approach for Machine Learning and other applications. However, the way it was taught was ineffective for learning (to me personally and I suspect to many others as well) regardless of our different backgrounds or lack of strong math skills.

Although the concept of PCA is rather simple, its computations and the interpretations & meanings of each transformation are challenging without examples and visualizations. We did have some examples in the course, but not detailed and complete enough to fully understand PCA intuitively and thoroughly.

One ended up spending huge amount of time reading additional relevant articles on internet, fishing out useful information from Discussion Forums, yet still struggling a great deal to write codes for completing assignments. Even though the codes run through without returning mistakes and one gets "pass", there is not much satisfaction and joy one feels in learning because sometimes, the initial learning from listening to the lectures was so insufficient and inadequate for doing the assignment that one simply completes the task doing all one can without further understanding and solidification of the knowledge.

This feeling is in such a huge contrast to those when learning the two other related courses in the Specialization (Linear Algebra and Multivariate Calculus) during which one enjoyed immensely the learning process as well as the content. Here I do not think that one can attribute the difference to the seemingly higher level of difficulty of PCA. The instructors for those two courses did an excellent job to explain things intuitively and thoroughly.

Marc is an extremely knowledgeable and competent professional. By changing the teaching style more in tune with the learners, I am sure more people will learn a great deal from this course and enjoy the learning process in the meantime. Thank you for the hard work put into teaching this course all the same.

p.s. If the teaching staff involved can fix all technical issues or mistakes related to the Assignments, it will greatly benefit the learners. Thank you.

By Akiva K S

•Jun 13, 2020

I passed the course with good grades, I like an idea of such course. But my opinion is: the course needs substantial improvement. Period. I personally enjoyed listening to Marc Peter - he's an excellent lecturer and super-smart guy. His book on math for Machine Learning is challenging, but almost perfect. But the course itself is a disappointment. 1) Precious lecturer's time is _wasted_ on explaining very basic concepts such as mean and variance... to make the course accessible for poor gals/guys with no math in head at all and, consequently, to enable Coursera to earn more $$. But it doesn't help - what they'll do in Week 4 once eigenvectors with show up from nowhere?? Unfortunately the course is not for them

2) Lecturer wastes his _super precious_ time by multiplying matrices by hand. Screw it. I'm also lecturer at university and from my experience such demo should be done once/twice. And after that, guys, matrix operations in numpy have to be demonstrated in the class, otherwise practical exercises could be done only by those with solid prior experience in Python + NumPy

3) Quality of practical assignments is below any critics. Some cannot work at Coursera platform, they should be run locally and to run Jupiter Notebook locally one has to be seasoned Python programmer and good DevOps. Guidelines to practical assignments do not guide at all. SW practices in assignments are dubious.

Bottom line: kindly advice to develop two courses - overview for those without linear algebra knowledge at all, and normal one - focused on Week 4 material. Coursera format with 5 minute lectures cannot accommodate such course? Leave this platform, Marc Peter is great lecturer and specialist, his name should not be associated with such failure.

Regards,

Akiva

By Amar D N

•May 30, 2020

I have already completed this course but i felt like i needed to share my frustration regarding this 3rd course of the specialization. First of all, the previous two courses were excellent! I am not judging based on difficulty, those two courses opened my eyes on linear algebra and calculus. But this 'PCA' one is utterly disappointing. It revisited some theories of the previous courses in such a bad way.

If most of the things need to be learnt through the reading materials then is it justified to do this course? I mean I can find even better reading materials on the web. The only reason i kept on going is to go through the PCA portion of week 4. All topics of previous weeks were already covered by me that's why i didn't have to struggle much. But the explanations were quite inadequate and proofs of the theorems felt like rushed. I somehow managed to reach the final assignment and then my real frustration began. The grader was giving inappropriate results, submitting my code gave me 2/3 out of 10. after resubmitting with the same code multiple times, I finally passed the assignment. Won't recommend this course to anyone.

By Paul

•Aug 5, 2020

The lecturer doesn't provide us with adequate information to connect all materials together. Much knowledge in lectures are not enough or unrelated to completing programming homework or tests. In one of the programming assignment, the guidance is misleading and I believe most of us reached out for help in forum to finish the assignment.

In week one, everything looks pretty simple but the question and lectures are kind of unclear. Everything then goes worse in week two. the programming assignment is a nightmare. If you are familiar with python, you may only spend hours to complete this. Otherwiese you may have to struggle between unclear guidance, bugs and python syntax. Week three is not so bad but a lot of resource are from wikipedia with little illustration. Week four is jammed with auxiliary materials and massive critical PCA knowledge with limited explanation. I simply feel I was fooled to enroll in this course.

By Rachel S

•Jul 9, 2019

After the first two courses in the specialisation, this one was truly disappointing. You are warned at the beginning that this course is challenging. This is true, but there is absolutely no reason why it should be THIS challenging. There are several factors that make this course more difficult than it needs to be. The poor pacing leads to a bizarre mix of repetitive trivial questions and vague assignments with poor explanation and over-reliance on reading external sources. Nobody wants constant hand-holding but the lack of direction will lead to you wasting far too much time chasing down minor technical errors and figuring out what on earth is being asked of you. Finishing this course was a slog and I just wanted to wash my hands of it. The first two courses in this specialisation are great and I highly recommend them, but I would not be happy if I had paid £38 for this course.

By Mikhail D

•May 27, 2020

I really loved the first two courses in the specialisation, but this one honestly is a disaster. This is bad teaching at its finest: "I'll throw a bunch of formulas at you and it is your job to figure out what they mean", "Here is an important concept that is critical to understanding the material, but I don't have time to cover it so please check it out Wikipedia instead".

The lecturer shows no passion to the subject whatsoever and spends all the time writing out monstrous formal definitions instead of trying to build student's intuition of what things really mean. This is exactly what Sam and David were so good at in the first two courses, and it is a real shame they had to replace them for this final course.

As others pointed out programming assignments are indeed poorly constructed, with lots of pitfalls and generally speaking very frustrating.

By James P

•Jun 10, 2018

After taking/passing the two previous courses, this course is very disappointing. The programming assignments are more about numpy/python peculiarities (which dimension is D or N) and deciphering cryptographic specifications (X is documented as an input but not a parameter to the function). The misleading templates appear to be intentional - it is not clear what educational purpose this serves. The difficulty in this course is not conceptual understanding - it is difficult because the assignments are intentionally confusing. Another point regarding programming in general. This course prefers implementing numerous functions (no testing), generating large amounts of random data as input, and assuming all goes well. Perhaps each function should be tested for correctness individually with known input/output - this is not a novel idea.

By Gabriel W

•May 23, 2020

I did the 3 specialization lessons "Mathematics for Machine Learning" (Linear Algebra, Multivariate Calculus, PCA). I really had a lot of fun and learnings in the first one (5 stars for Linear Algebra): David Dye is an increadible teacher. The second one is okay (3 stars for me). In the third one (PCA) the expected knowledge difference between the lessons (easy to follow) and the programming tasks of weeks 2 and 4 was to high and to much challenging for me. I had no fun to pass the corresponding tests and I have finished the lessons with the only one target to be done. It doesn't correspond to what I'm looking for when I'm learning during my week-end.

By Nathan R

•Jan 22, 2020

This was a terrible course in every way possible. DO NOT waste your time and money on it. The lecturer skips over things way too fast and delivers poor explanations, and then gives ridiculously hard programming assignments when this course is supposed to be mainly about maths. Moreover, he asks quiz questions about topics he doesn't even cover in the lectures, and the answers provided are terrible. Very poor quality course, which is a shame, because the other two courses in this specialization are actually worth doing.

By Naveen K

•Aug 9, 2018

I've finished all the two previous courses in this specialization.I was shocked at seeing the content and programming assignments given to us.It was totally different.They expect a lot from us.Content is not up to the mark.First two courses was awesome.But this course is an exact opposite to the first two.Totally disappointed!! I was hoping to finish this specialization.But it seems I cannot. I didn't expect this.

By Ong J R

•Aug 11, 2018

Concepts weren't taught well and programming exercises are full of errors. Very difficult to debug and find out if I am on track during the programming exercises. Lecturer lacks passion and ability to convey core concepts well to audience. Hard to follow up on the mathematical derivation with the simple stuff that we were taught in module 1 and 2.

By Steve

•Sep 5, 2020

Very sketchy presentation of complex material. Each lecture averaged around 5-6 minutes when they should have been 15-20 minutes. As a result the instructor glossed over the material without adequate explanations and derivations. And no one at Imperial College seems to be responding to recent posts in the discussion forums.

By Valeria B

•Jun 26, 2019

Too few examples given during the lessons. More examples could greatly improve understanding and the solution of quizzes and programming assignment.

I had to integrate this course with multiple sources I looked up for by myself, so I'm really wondering if I wisely spent my money on this course.

By Yaroshchuk A

•May 22, 2020

Instructor writes down equations and formal definitions while reading out loud what he is writing. None further explanations are given.

Basically whole course is a voiced list of equations together with some links to Wikipedia which even further empathize pathetic quality of content.

By Alisa G

•Jul 23, 2020

The lectures are only partly related to the quizzes and assignments, some parts are just unnecessarily over complicated and confusing. The final and most important assignment is so computationally heavy so it's hardly running locally

By Tathagat A

•Jun 15, 2020

The lecturer was not always understandable.

By Israel J L

•Jan 6, 2019

Great course !! Definitely it's an intermediate course so if you don't have a college level in lineal algebra and calculus you'll struggle with the videos and the notebooks (besides you need basic level programing in python and numpy)

The videos are kinda hard but it seems that Marc it's a great mathematician and also he shares a great e-book written by him that has everything seen in the course and more, so with this you can get all the knowledge need it to understand PCA.

I don't understand why it's only 4 stars rated; again if you want to learn linear algebra and calculus, this is not the place... you need to have the needed level to suceed.

By Veeramani. S

•Sep 6, 2020

Good Explanation. Very helpful for learning an application of mathematics through this course

By Makozz Q

•Jul 5, 2020

I'd like to say thanks to everyone who has made this learning experience possible.

Thank you, Marc. Your explanations combined with the book "Mathematics for Machine Learning" have come really handy.

It has been an amazing journey to see how linear algebra marries multivariate calculus to give birth to to PCA.

Being a linguist, I must admit I'm quite new to Python and the domain of machine learning. It would be great if you could add some polishing touches to the programming assignments, especially the one in Week 4 (PCA): waiting for a long time until the system finishes crunching the code was quite a slow experience. If that has to do with a student's sloppy code, please add some recommendations inside the assignment on how to avoid this trap. If that is caused by some technical issues on the server side, please take a moment to look at this.

That you have added the Python tutorial is really helpful. Could you also consider updating it with some details on how to sort eigenvectors and eigenvalues to collect these into a covariance matrix. This piece was mighty tough.

Thank you once again. Keep on!

By Henry N

•Aug 27, 2020

Overall this was a pretty good course - some other reviews comment on how some things are glossed over in the videos but you'll get the most out of it if the other courses in the specialisation are fresh in your mind (e.g. you'll have to know about eigenvectors/eigenvalues, Gaussian elimination, derivatives and the chain rule etc. as these are referred to and used but not explained in detail - but these are covered in the first 2 courses). The main problem is with the assignments - for some weeks there's not enough guidance about what the functions should be returning, so these should be better documented; the other issue is that some of the code that we are not required to edit doesn't actually work - for instance, my implementation of PCA passed the grader but the visualisations in the week 4 notebook didn't work.

- Finding Purpose & Meaning in Life
- Understanding Medical Research
- Japanese for Beginners
- Introduction to Cloud Computing
- Foundations of Mindfulness
- Fundamentals of Finance
- Machine Learning
- Machine Learning Using Sas Viya
- The Science of Well Being
- Covid-19 Contact Tracing
- AI for Everyone
- Financial Markets
- Introduction to Psychology
- Getting Started with AWS
- International Marketing
- C++
- Predictive Analytics & Data Mining
- UCSD Learning How to Learn
- Michigan Programming for Everybody
- JHU R Programming
- Google CBRS CPI Training

- Natural Language Processing (NLP)
- AI for Medicine
- Good with Words: Writing & Editing
- Infections Disease Modeling
- The Pronounciation of American English
- Software Testing Automation
- Deep Learning
- Python for Everybody
- Data Science
- Business Foundations
- Excel Skills for Business
- Data Science with Python
- Finance for Everyone
- Communication Skills for Engineers
- Sales Training
- Career Brand Management
- Wharton Business Analytics
- Penn Positive Psychology
- Washington Machine Learning
- CalArts Graphic Design

- Professional Certificates
- MasterTrack Certificates
- Google IT Support
- IBM Data Science
- Google Cloud Data Engineering
- IBM Applied AI
- Google Cloud Architecture
- IBM Cybersecurity Analyst
- Google IT Automation with Python
- IBM z/OS Mainframe Practitioner
- UCI Applied Project Management
- Instructional Design Certificate
- Construction Engineering and Management Certificate
- Big Data Certificate
- Machine Learning for Analytics Certificate
- Innovation Management & Entrepreneurship Certificate
- Sustainabaility and Development Certificate
- Social Work Certificate
- AI and Machine Learning Certificate
- Spatial Data Analysis and Visualization Certificate

- Computer Science Degrees
- Business Degrees
- Public Health Degrees
- Data Science Degrees
- Bachelor's Degrees
- Bachelor of Computer Science
- MS Electrical Engineering
- Bachelor Completion Degree
- MS Management
- MS Computer Science
- MPH
- Accounting Master's Degree
- MCIT
- MBA Online
- Master of Applied Data Science
- Global MBA
- Master's of Innovation & Entrepreneurship
- MCS Data Science
- Master's in Computer Science
- Master's in Public Health