Back to Mathematics for Machine Learning: PCA

stars

3,051 ratings

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.
At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If youâ€™re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge.
The lectures, examples and exercises require:
1. Some ability of abstract thinking
2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis)
3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization)
4. Basic knowledge in python programming and numpy
Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms....

WS

Jul 6, 2021

Now i feel confident about pursuing machine learning courses in the future as I have learned most of the mathematics which will be helpful in building the base for machine learning, data science.

JS

Jul 16, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

Filter by:

By Hashaam S

â€¢Dec 30, 2018

This course does not include real-world examples as compared with other 2 courses in the series. The other lecturers energy was quite obvious and the lecturer in this course does nothing extra. This is the traditional teaching method where we should keep on grasping theory without understanding applications.

While I have learned many new concepts, I am 100% unsure I have not understood the course in general.

The assignments in the course was poorly constructed and the lectures did not explain anything more about numpy methods.

I will not recommend this course to anyone.

By Maximilian W

â€¢Apr 29, 2019

The first two courses in the Mathematics for Machine Learning specialisation are excellent - even amongst the best online or traditional maths courses I have taken. This course was seriously lacking. Not in content, or even the ability of the lecturer, but rather in how the information is conveyed. There are some excellent reviews which elaborate further in to the problems with this course, so I will not labour over them all. In essence, if you are learning in your own free time, the poorer information transfer is not appreciated.

However, this course is important, but if you are unsure of whether or not to invest your time into starting this course (now) , I think the following questions are good to ask. Are you:

1) fairly competent in maths, at least significantly beyond the first two courses. This is not because the underlying maths is hard, but the way the information is conveyed, will require more firm knowledge, or, are you:

2) willing to be frustrated, and grab additional resources. You need to be patient to get the most out of this course. The previous courses were great at guiding, and in large part spoon feeding. This course is different, and you have to be happy with that.

3) proficient at numpy and python. I would invest time before the course working on basic numpy skills, as this will make the assignments much easier, and allow you to focus on implementation of learning rather than debugging, and pulling out of hair.

The two star review is because this course didn't provide the high quality expected from the first courses, however the content and end learning result can not be questioned as poor.

By Eric P

â€¢Apr 26, 2019

There is little reason to take this course except for gaining the satisfaction of completing the three courses in the series. There are briefer, more satisfying introductions to PCA elsewhere. This course has too little of what made the other courses in the series so good and shares too much of their shortcomings. Where the other two courses excelled in demonstrating an intuitive understanding of both the maths and their applications, this course really avoids all effort at intuition or examples and instead just throws formula after formula at you. You are then given programming assignments where at least half the effort is to try to understand what is being asked before you start to work to implement it. This leaves you more with a feeling of only having completed assignments and less a sense that youâ€™ve developed a capability in either the maths or their applications. In the end, I am left with a strong desire to learn more about the maths of PCA and their application only because I am eager to hear the subject matter explained by someone else.

The other two courses demonstrated the potential of how good e-learning can be. This course is just another example of its shortcomings.

By Vyacheslav T

â€¢Mar 24, 2019

Algebra course is excellent. Calculus course is good. PCA is so bad that I am still upset that I spent my time on it.

By Christos M

â€¢Apr 27, 2019

Unfortunately this course does is of much lower quality than the previous courses of the specialization. There is no progression towards the assignments which basically ask you to implement something without any context. There was even a technical issue with the grader for the first assignment.

If you want to complete it to finish the specialization, you need to seek help in the forums as there are a lot of helpful answers.

By Avirup G

â€¢Feb 18, 2019

Very poorly written/performed material with inadequate coding help. The engagement level is quite low. Will not recommend if you have novice programming background or new to math concepts.

By Alexandra S

â€¢Sep 26, 2018

Worth auditing because the video lessons are good but unless you have solid Python programming experience, the assignments and some quiz questions will take you days instead of hours. The course info states that you would need 'Basic knowledge in python programming and numpy'. This is to be understood as 'solid practice, at least intermediate level'. 'Basic knowledge' simply does not make it possible to finish these exercises within the given timeframe (1-2 or even 3-4 hours).

As many others, despite having no problem with the maths, I gave up during week 1 because of this issue and of not having this amount of free time while already working full time. This should be emphasised in the course description, apart from the extra mathematical creativity that already appears there. And it should also appear in the description of the whole specialisation, which states that programming experience 'comes in handy but is not necessary'. Many people who start doing the courses are determined to finish the whole specialisation.

By Bryan S

â€¢Feb 19, 2019

This course needs a lot of work to get to the level of quality of its two predecessors.

By Sreekar P

â€¢Oct 23, 2018

The instruction is absolutely bad and not worth it. However, if you have patience to re-watch, refer to other supporting materials, learn on your own a lot and then have patience to deal with programming asssignments ,...then you might find the final result useful.

By Harshit D

â€¢Jul 30, 2018

Loved the first two courses but felt like killing myself in this course. One of the worst professors i have ever encountered.

By Guillermo A

â€¢Jun 15, 2020

I agree with most other reviewers that this course is more difficult than the previous two in the specialization. However, I disagree with some of the very negative reviews. I believe the instructor does generally a good job (and is clearly extremely knowledgeable and competent) except:

(1) Sometimes, in the derivations, he glosses over steps which are probably trivial to him, but that are not so trivial to many in the audience (in my case, I had to replay some of these derivation steps a few times until I could understand them)

(2) As some reviewers have said, a few more detailed examples here and there (as opposed to quickly flashing some sample data plots) would be helpful

(3) The instructor could use a little bit more of charisma to try to show that he is excited about the subject he is teaching, and thus make his audience more excited about it (but granted, that is a personality trait that doesn't come easy to most professors and instructors).

As for the programming exercises, which have been amply criticized in other reviews, I agree there's little guidance on how to approach them, but for anybody with who has understood the lectures fairly well, and with enough programming experience, they should not be that hard. It's only a few lines of code that need to be written in all of the programming assignments.

By Brock I

â€¢Nov 21, 2018

Way too hard compared to the other courses in the specialization. I feel like I wasted my money on this.

By Roy A

â€¢Sep 23, 2020

First 2 courses in this specialization was really good, so I'm very dissappointed here. My main issues are the following:

The lectures are not clear enough, for about 75% of the assignments, I had to look up alternative lectures on youtube to get the point, if I have to find the majority of the information outside of Coursera, then what is the point of the course?

Sometimes there are no examples in the lectures, other times, the examples are too basic. So once I got a more advanced question , I was clueless how to solve it. Sometimes the first question on a quiz is much more complex than the example in the lecture.

The lectures requires you to be very comfortable with math notation, which I'm not. As mentioned earlier, some more examples would have solved this for me, but as the examples are lacking, I was simply unable to understand what was being written on the lecture, and had to look elsewhere. Note that the math itself wasn't hard(once I found someone else to explain it), just understanding what the lecturer meant.

Some steps in lectures are missing, I guess they are obvious if you have the correct prerequisites, but to me it was just a black box. "We have x,y,x. So then we get x+z,y!", why? no idea, I still don't know why we get x+z.

Programming assignments are hard for the wrong reasons, the math is not that hard, the python and numpy is basic, but the explanations of what the function is supposed to do is not clear. When I got stuck, it was usually because I didn't understand what the output of a function was supposed to be. As this is a math course, I would expect the challenge to be the math, not something else.

To sum up the above points, I think the course lacks a good understanding of the base skills needed to complete the course. Since I had no problems with part 1 and 2, and then ran into a wall at part 3(PCA), I think these parts should be better synced, if they are to be part of the same specialization.

By Rahul M

â€¢Jun 29, 2019

The instruction content was superb, though the tests were unrelated to content. I realize that there is a lot of criticism of the course here, but if you understand and code in Python, they are not hard. I wish the course staff had made the Jupyter notebooks more clear, giving us an understanding of what was required, if that was done this is a good, though very hard (in terms of mathematics) course.

By JoÃ£o S

â€¢May 2, 2019

It is a good course but some problems must be reported. Despite the previous courses from the specialization, I missed the conceptual explanations, the development of intuitive understanding. The support is almost inexistent: questions on forums are not answered by lecturers or mentors, some programming exercises requires knowledge not even mentioned on classes and I feel it is a non necessary knowledge at all to the purpose of the course. Some tutorials would help. Only other students make things clearer at some points. Some lectures have "magic passes" not explained, specially on PCA subject itself, week 4. Maybe the courser could have a additional week to teach things in a better way.

By Nimesh S

â€¢Jun 19, 2020

Relatively tougher than previous two courses in the specialization. I'd suggest giving more time and being patient in pursuit of completing this course and understanding the concepts involved.

By Martin B

â€¢Oct 22, 2018

Overall: worthwhile content, but poor execution. Especially assignments need improvement.

Good points:

-The contents tend to be worthwhile.

-The instructor is thorough and clear.

Bad Points :

-To those who are not as familiar with mathematical terminology the instructor is a tough act to follow sometimes.

-The great disappointment of this course lies in the assignments. They don't really add to my understanding of the mathematics involved, and are quite often a distraction because the assignments are quite inflexible in terms of coding: you'll have to stick quite close to what the instructor envisions, or you will fail. This is especially frustrating because you will have a hard time figuring out whether you failed because your code was faulty or because your conceptual understanding was faulty.

By Jong H S

â€¢Jul 17, 2018

This is one hell of an inspiring course that demystified the difficult concepts and math behind PCA. Excellent instructors in imparting the these knowledge with easy-to-understand illustrations.

By Oliverio J S J

â€¢May 29, 2020

This course is awful. The videos have no useful explanations, the speaker seems to be reading some slides. The provided material is really bad; there are even links to wikipedia! The difficulty level of the assignments is beyond the one proposed by the lessons. The programming tasks consist only on reproducing formulas; most of the time you are struggling with numpy implementation issues. In summary: stay away from this course.

By Thomas B

â€¢Jun 4, 2022

While the first two courses in the specialization were excellent, the negative reviews here are on-point as this course was a terrible experience. The prerequites don't match the course, or those of the previous courses. The course heavily uses formal mathematical language that won't be familiar to those who haven't at least taken a few upper-level university math courses. (The stated Python/numpy prerequisites are also inaccurate, but this did not trip me up as much.) Many of the concepts covered are purely academic, such as the formal definition of an inner product; there is little-to-no practical application of these concepts, meaning that much of the course is only useful if you have an interest in pure maths. Unfortunately, given that the specialization focuses on applying these concepts to machine learning, this means that the course barely helps the average learner who wants to learn more about machine learning. The assignments are very poorly implemented. Like others have mentioned, the instructions are unclear and there are possible bugs within the tests themselves. Comments sometimes are sloppily copy/pasted between code blocks, referencing parameters that don't exist or giving an instruction that should not actually be done. The facial recognition and script recognition themes of the assignments are really cool, but the poor writing in the assignments make it really unclear how the maths you implement relate to those themes, totally undercutting any practical illustration you may hope to gain. I'll add that the "tests" meant to help debug your code are almost pointless, because there is no feedback on why the tests are failing; that means that whenever a test fails, you may be stuck making random changes to your code in the hope that some arbitrary change will work.

The lack of a cheat sheet is cruel.

Looking back on the course, it is heavily padded. Weeks 2 and 3 either cover concepts that are not directly used in the PCA algorithm, or which are rehashes of material from the previous two courses. Students could theoretically learn about PCA simply by applying previously taught concepts about projections and eigenvectors to the covariance matrix introduced in week 1. Four weeks of material are simply not justified. PCA is not a difficult concept to wrap your head around if you have some understanding of linear algebra and basic data science. The previous courses made linear algebra and multivariate calculus approachable, practical, and fun. This course, on the other hand, is overcomplicated, light on useful material, artificially difficult, and made me want to pull out my hair.

By Christian R

â€¢Jul 24, 2018

Frustrating. Videos and material does not cover what it is asked for in the quizzes and assignments.

By JICHEN W

â€¢Oct 27, 2018

Explanation of course material is not clear

By Jayant V

â€¢May 1, 2018

This course was definitely a bit more complex, not so much in assignments but in the core concepts handled, than the others in the specialisation. Overall, it was fun to do this course!

By JosÃ© D

â€¢Oct 31, 2018

This course is harder that the the two first ones. You have to do a lot more by yourself. There will be some frustrations with assignments that are not always easy or clear, with confusing python/numpy notations not really introduced during the course. Also, most assignment didn't work online, so I had to install python3 and jupyter to work on them locally and submit them manually. You should expect to spend more time than announced. All in all, I've learned new things and that's the most important. I believe there are room for improvement for this course.

By Susan H M

â€¢Oct 21, 2019

Some people complained about this course or specialization such that there was not enough help or the instructors were so lazy that a lot of the time, they just rushed through and ignored the details. But I LIKE this course and this specialization. All those complaints expressed by some people worked to my advantages. I do not need someone to monitor me at every step, so I do not mind the instructors stay out of discussion forum most of the time. Indeed, the most recent reply from an instructor I saw was a year old (or 2-year old, don't remember exactly). I would have genuinely hated this course or specialization if the instructors intervened at every step.

Throughout this specialization, I was able to discuss course contents and algorithm codes freely on the discussion forum. That really enhanced the learning experiences. I did not feel that my learning journey was micromanaged and I truly valued that freedom to develop my own understanding and insights.

I gave all three courses in the specialization a five-star review (would give six stars if that is available). I would definitely take another course by the instructors on Coursera.