Back to Mathematics for Machine Learning: PCA

4.0

stars

2,285 ratings

•

574 reviews

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

JS

Jul 16, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

NS

Jun 18, 2020

Relatively tougher than previous two courses in the specialization. I'd suggest giving more time and being patient in pursuit of completing this course and understanding the concepts involved.

Filter by:

By Kathleen D

•Dec 10, 2020

This course covers critical material, but unfortunately does not present them well. I found myself having to constantly find outside resources for clearer explanations. For many topics, the course itself makes no attempt to explain them, but just presents links to dense reading material and, in a couple of cases, Wikipedia articles. In my opinion, if you're linking to Wikipedia for core components of the course, you're no longer justified charging for that course. The lectures are spent deriving formulas, but providing little or no intuition about what they mean or how they are used. The course does not prepare one well for the final lab in particular, which felt like a disconnect from the lectures, had unclear explanations of what to do, and contained errors in the provided code. I did power through it, and I did learn something about PCA, but feel like I needed there to be something more to cement what I learned in practical understanding.

By Sergii T

•Dec 22, 2018

Course is targeted more on pure math derivations, rather then real world applications. For my opinion, it doesn't fit well with other courses in this specialisation. it goes too Deep in math derivations. It should fit for students interested in mathematics and not engineers, who want to get more insights in ML related math.

By Roy A

•Sep 23, 2020

First 2 courses in this specialization was really good, so I'm very dissappointed here. My main issues are the following:

The lectures are not clear enough, for about 75% of the assignments, I had to look up alternative lectures on youtube to get the point, if I have to find the majority of the information outside of Coursera, then what is the point of the course?

Sometimes there are no examples in the lectures, other times, the examples are too basic. So once I got a more advanced question , I was clueless how to solve it. Sometimes the first question on a quiz is much more complex than the example in the lecture.

The lectures requires you to be very comfortable with math notation, which I'm not. As mentioned earlier, some more examples would have solved this for me, but as the examples are lacking, I was simply unable to understand what was being written on the lecture, and had to look elsewhere. Note that the math itself wasn't hard(once I found someone else to explain it), just understanding what the lecturer meant.

Some steps in lectures are missing, I guess they are obvious if you have the correct prerequisites, but to me it was just a black box. "We have x,y,x. So then we get x+z,y!", why? no idea, I still don't know why we get x+z.

Programming assignments are hard for the wrong reasons, the math is not that hard, the python and numpy is basic, but the explanations of what the function is supposed to do is not clear. When I got stuck, it was usually because I didn't understand what the output of a function was supposed to be. As this is a math course, I would expect the challenge to be the math, not something else.

To sum up the above points, I think the course lacks a good understanding of the base skills needed to complete the course. Since I had no problems with part 1 and 2, and then ran into a wall at part 3(PCA), I think these parts should be better synced, if they are to be part of the same specialization.

By Akiva K S

•Jun 13, 2020

I passed the course with good grades, I like an idea of such course. But my opinion is: the course needs substantial improvement. Period. I personally enjoyed listening to Marc Peter - he's an excellent lecturer and super-smart guy. His book on math for Machine Learning is challenging, but almost perfect. But the course itself is a disappointment. 1) Precious lecturer's time is _wasted_ on explaining very basic concepts such as mean and variance... to make the course accessible for poor gals/guys with no math in head at all and, consequently, to enable Coursera to earn more $$. But it doesn't help - what they'll do in Week 4 once eigenvectors with show up from nowhere?? Unfortunately the course is not for them

2) Lecturer wastes his _super precious_ time by multiplying matrices by hand. Screw it. I'm also lecturer at university and from my experience such demo should be done once/twice. And after that, guys, matrix operations in numpy have to be demonstrated in the class, otherwise practical exercises could be done only by those with solid prior experience in Python + NumPy

3) Quality of practical assignments is below any critics. Some cannot work at Coursera platform, they should be run locally and to run Jupiter Notebook locally one has to be seasoned Python programmer and good DevOps. Guidelines to practical assignments do not guide at all. SW practices in assignments are dubious.

Bottom line: kindly advice to develop two courses - overview for those without linear algebra knowledge at all, and normal one - focused on Week 4 material. Coursera format with 5 minute lectures cannot accommodate such course? Leave this platform, Marc Peter is great lecturer and specialist, his name should not be associated with such failure.

Regards,

Akiva

By Amar D N

•May 30, 2020

I have already completed this course but i felt like i needed to share my frustration regarding this 3rd course of the specialization. First of all, the previous two courses were excellent! I am not judging based on difficulty, those two courses opened my eyes on linear algebra and calculus. But this 'PCA' one is utterly disappointing. It revisited some theories of the previous courses in such a bad way.

If most of the things need to be learnt through the reading materials then is it justified to do this course? I mean I can find even better reading materials on the web. The only reason i kept on going is to go through the PCA portion of week 4. All topics of previous weeks were already covered by me that's why i didn't have to struggle much. But the explanations were quite inadequate and proofs of the theorems felt like rushed. I somehow managed to reach the final assignment and then my real frustration began. The grader was giving inappropriate results, submitting my code gave me 2/3 out of 10. after resubmitting with the same code multiple times, I finally passed the assignment. Won't recommend this course to anyone.

By Paul

•Aug 4, 2020

The lecturer doesn't provide us with adequate information to connect all materials together. Much knowledge in lectures are not enough or unrelated to completing programming homework or tests. In one of the programming assignment, the guidance is misleading and I believe most of us reached out for help in forum to finish the assignment.

In week one, everything looks pretty simple but the question and lectures are kind of unclear. Everything then goes worse in week two. the programming assignment is a nightmare. If you are familiar with python, you may only spend hours to complete this. Otherwiese you may have to struggle between unclear guidance, bugs and python syntax. Week three is not so bad but a lot of resource are from wikipedia with little illustration. Week four is jammed with auxiliary materials and massive critical PCA knowledge with limited explanation. I simply feel I was fooled to enroll in this course.

By Rachel S

•Jul 9, 2019

After the first two courses in the specialisation, this one was truly disappointing. You are warned at the beginning that this course is challenging. This is true, but there is absolutely no reason why it should be THIS challenging. There are several factors that make this course more difficult than it needs to be. The poor pacing leads to a bizarre mix of repetitive trivial questions and vague assignments with poor explanation and over-reliance on reading external sources. Nobody wants constant hand-holding but the lack of direction will lead to you wasting far too much time chasing down minor technical errors and figuring out what on earth is being asked of you. Finishing this course was a slog and I just wanted to wash my hands of it. The first two courses in this specialisation are great and I highly recommend them, but I would not be happy if I had paid £38 for this course.

By Mikhail D

•May 27, 2020

I really loved the first two courses in the specialisation, but this one honestly is a disaster. This is bad teaching at its finest: "I'll throw a bunch of formulas at you and it is your job to figure out what they mean", "Here is an important concept that is critical to understanding the material, but I don't have time to cover it so please check it out Wikipedia instead".

The lecturer shows no passion to the subject whatsoever and spends all the time writing out monstrous formal definitions instead of trying to build student's intuition of what things really mean. This is exactly what Sam and David were so good at in the first two courses, and it is a real shame they had to replace them for this final course.

As others pointed out programming assignments are indeed poorly constructed, with lots of pitfalls and generally speaking very frustrating.

By James P

•Jun 10, 2018

After taking/passing the two previous courses, this course is very disappointing. The programming assignments are more about numpy/python peculiarities (which dimension is D or N) and deciphering cryptographic specifications (X is documented as an input but not a parameter to the function). The misleading templates appear to be intentional - it is not clear what educational purpose this serves. The difficulty in this course is not conceptual understanding - it is difficult because the assignments are intentionally confusing. Another point regarding programming in general. This course prefers implementing numerous functions (no testing), generating large amounts of random data as input, and assuming all goes well. Perhaps each function should be tested for correctness individually with known input/output - this is not a novel idea.

By Gabriel W

•May 23, 2020

I did the 3 specialization lessons "Mathematics for Machine Learning" (Linear Algebra, Multivariate Calculus, PCA). I really had a lot of fun and learnings in the first one (5 stars for Linear Algebra): David Dye is an increadible teacher. The second one is okay (3 stars for me). In the third one (PCA) the expected knowledge difference between the lessons (easy to follow) and the programming tasks of weeks 2 and 4 was to high and to much challenging for me. I had no fun to pass the corresponding tests and I have finished the lessons with the only one target to be done. It doesn't correspond to what I'm looking for when I'm learning during my week-end.

By Nathan R

•Jan 22, 2020

This was a terrible course in every way possible. DO NOT waste your time and money on it. The lecturer skips over things way too fast and delivers poor explanations, and then gives ridiculously hard programming assignments when this course is supposed to be mainly about maths. Moreover, he asks quiz questions about topics he doesn't even cover in the lectures, and the answers provided are terrible. Very poor quality course, which is a shame, because the other two courses in this specialization are actually worth doing.

By Naveen K

•Aug 9, 2018

I've finished all the two previous courses in this specialization.I was shocked at seeing the content and programming assignments given to us.It was totally different.They expect a lot from us.Content is not up to the mark.First two courses was awesome.But this course is an exact opposite to the first two.Totally disappointed!! I was hoping to finish this specialization.But it seems I cannot. I didn't expect this.

By Ong J R

•Aug 11, 2018

Concepts weren't taught well and programming exercises are full of errors. Very difficult to debug and find out if I am on track during the programming exercises. Lecturer lacks passion and ability to convey core concepts well to audience. Hard to follow up on the mathematical derivation with the simple stuff that we were taught in module 1 and 2.

By Valeria B

•Jun 26, 2019

Too few examples given during the lessons. More examples could greatly improve understanding and the solution of quizzes and programming assignment.

I had to integrate this course with multiple sources I looked up for by myself, so I'm really wondering if I wisely spent my money on this course.

By Yaroshchuk A

•May 22, 2020

Instructor writes down equations and formal definitions while reading out loud what he is writing. None further explanations are given.

Basically whole course is a voiced list of equations together with some links to Wikipedia which even further empathize pathetic quality of content.

By Alisa G

•Jul 23, 2020

The lectures are only partly related to the quizzes and assignments, some parts are just unnecessarily over complicated and confusing. The final and most important assignment is so computationally heavy so it's hardly running locally

By Tathagat A

•Jun 15, 2020

The lecturer was not always understandable.

By Israel J L

•Jan 6, 2019

Great course !! Definitely it's an intermediate course so if you don't have a college level in lineal algebra and calculus you'll struggle with the videos and the notebooks (besides you need basic level programing in python and numpy)

The videos are kinda hard but it seems that Marc it's a great mathematician and also he shares a great e-book written by him that has everything seen in the course and more, so with this you can get all the knowledge need it to understand PCA.

I don't understand why it's only 4 stars rated; again if you want to learn linear algebra and calculus, this is not the place... you need to have the needed level to suceed.

By Veeramani. S

•Sep 6, 2020

Good Explanation. Very helpful for learning an application of mathematics through this course

By Makozz Q

•Jul 5, 2020

I'd like to say thanks to everyone who has made this learning experience possible.

Thank you, Marc. Your explanations combined with the book "Mathematics for Machine Learning" have come really handy.

It has been an amazing journey to see how linear algebra marries multivariate calculus to give birth to to PCA.

Being a linguist, I must admit I'm quite new to Python and the domain of machine learning. It would be great if you could add some polishing touches to the programming assignments, especially the one in Week 4 (PCA): waiting for a long time until the system finishes crunching the code was quite a slow experience. If that has to do with a student's sloppy code, please add some recommendations inside the assignment on how to avoid this trap. If that is caused by some technical issues on the server side, please take a moment to look at this.

That you have added the Python tutorial is really helpful. Could you also consider updating it with some details on how to sort eigenvectors and eigenvalues to collect these into a covariance matrix. This piece was mighty tough.

Thank you once again. Keep on!

By Henry N

•Aug 27, 2020

Overall this was a pretty good course - some other reviews comment on how some things are glossed over in the videos but you'll get the most out of it if the other courses in the specialisation are fresh in your mind (e.g. you'll have to know about eigenvectors/eigenvalues, Gaussian elimination, derivatives and the chain rule etc. as these are referred to and used but not explained in detail - but these are covered in the first 2 courses). The main problem is with the assignments - for some weeks there's not enough guidance about what the functions should be returning, so these should be better documented; the other issue is that some of the code that we are not required to edit doesn't actually work - for instance, my implementation of PCA passed the grader but the visualisations in the week 4 notebook didn't work.

By Andrea V

•Jun 22, 2019

This course is hard, and contains a lot of mathematical derivations and concepts that might be overwhelming for somebody not completely fresh in maths. Nevertheless, it offers a good balance between rigour and practical application, and if some lectures turn out to be too complicated, there's always the chance to deepen the matter more quitely using the course material or online resources. I think that the course would have benefited from a more aneddoctical approach at times: for instance restating in english what the general purpose of PCA is, could help the less mathematically inclined to better seize the idea. But I know this is not always easy to do.

By Arka S

•May 27, 2020

Frankly, after the high of the first two courses of this specialisation, this one was a low. Instruction was typical of most Universities; heavily analytical and monotonous. This was not a proper way, especially for such a complicated (for beginners) topic like PCA. This course could've been executed in a much better way.

Still a lot of insight is there to be gained, and I learnt quite a few things. The simplification of the cost (or loss) function was explained well, and I had quite a few 'Aha!' moments in this course as well (in Weeks 3 and 4), albeit not as much as I did in the first two courses (Lin Alg and Multivariate Calc).

By Ruarob T

•Jun 30, 2019

Make sure you have time and be ready for python code debug. If you are just an average programmer with limited python exposure like me. It will take you a day to complete the programming assignment.

Note: the assignment and class VDO seems a distant - google a lot during the assignment/quiz

Note: Programming has little clue - personally, I think I spend so much time on programming (distracting me away from going back to Math review)

By Berkay E

•Aug 9, 2019

-Some of the contents are not clear.

+It gets great intuition for new learners in machine learning.

- Finding Purpose & Meaning in Life
- Understanding Medical Research
- Japanese for Beginners
- Introduction to Cloud Computing
- Foundations of Mindfulness
- Fundamentals of Finance
- Machine Learning
- Machine Learning Using Sas Viya
- The Science of Well Being
- Covid-19 Contact Tracing
- AI for Everyone
- Financial Markets
- Introduction to Psychology
- Getting Started with AWS
- International Marketing
- C++
- Predictive Analytics & Data Mining
- UCSD Learning How to Learn
- Michigan Programming for Everybody
- JHU R Programming
- Google CBRS CPI Training

- Natural Language Processing (NLP)
- AI for Medicine
- Good with Words: Writing & Editing
- Infections Disease Modeling
- The Pronounciation of American English
- Software Testing Automation
- Deep Learning
- Python for Everybody
- Data Science
- Business Foundations
- Excel Skills for Business
- Data Science with Python
- Finance for Everyone
- Communication Skills for Engineers
- Sales Training
- Career Brand Management
- Wharton Business Analytics
- Penn Positive Psychology
- Washington Machine Learning
- CalArts Graphic Design

- Professional Certificates
- MasterTrack Certificates
- Google IT Support
- IBM Data Science
- Google Cloud Data Engineering
- IBM Applied AI
- Google Cloud Architecture
- IBM Cybersecurity Analyst
- Google IT Automation with Python
- IBM z/OS Mainframe Practitioner
- UCI Applied Project Management
- Instructional Design Certificate
- Construction Engineering and Management Certificate
- Big Data Certificate
- Machine Learning for Analytics Certificate
- Innovation Management & Entrepreneurship Certificate
- Sustainabaility and Development Certificate
- Social Work Certificate
- AI and Machine Learning Certificate
- Spatial Data Analysis and Visualization Certificate

- Computer Science Degrees
- Business Degrees
- Public Health Degrees
- Data Science Degrees
- Bachelor's Degrees
- Bachelor of Computer Science
- MS Electrical Engineering
- Bachelor Completion Degree
- MS Management
- MS Computer Science
- MPH
- Accounting Master's Degree
- MCIT
- MBA Online
- Master of Applied Data Science
- Global MBA
- Master's of Innovation & Entrepreneurship
- MCS Data Science
- Master's in Computer Science
- Master's in Public Health