Back to Mathematics for Machine Learning: Multivariate Calculus

stars

5,573 ratings

This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future....

JT

Nov 12, 2018

Excellent course. I completed this course with no prior knowledge of multivariate calculus and was successful nonetheless. It was challenging and extremely interesting, informative, and well designed.

DP

Nov 25, 2018

Great course to develop some understanding and intuition about the basic concepts used in optimization. Last 2 weeks were a bit on a lower level of quality then the rest in my opinion but still great.

Filter by:

By Ahmad H N

•Mar 16, 2021

Good

By Habib B K

•Mar 12, 2021

Nice

By Indah D S

•Feb 27, 2021

cool

By RAGHUVEER S D

•Jul 25, 2020

good

By Nat

•Mar 6, 2020

goot

By Zhao J

•Sep 11, 2019

GOOD

By Harsh D

•Jun 26, 2018

good

By Amini D P S

•Mar 26, 2022

wow

By Roberto

•Mar 25, 2021

thx

By Artem G

•May 28, 2022

:)

By Angel E E V

•Nov 30, 2021

:)

By Omar D

•May 5, 2020

gd

By Гончарова П В

•May 10, 2022

2

By Aidana P B

•Apr 26, 2021

щ

By Bhargava g

•Aug 7, 2020

.

By Kaushal K K

•Apr 23, 2022

A good, brief overview of the topics in multivariate calculus relevant to machine learning and optimisation. It may not necessarily go deep enough to make you an expert in solving problems in multivariate calculus that might be seen at the university level; rather, it goes just deep enough to enable you to understand how multivariable calculus operates in various machine learning scenarios. Some of these scenarios include:

(1) The process of backpropagation in basic neural networks.

(2) Using the Newton-Raphson method to find the roots of a function in the multivariate case.

(3) Use of the Taylor series to approximate a function in the multivariate case, and how such an approximation can be used for optimisation.

(4) Using gradient descent to reach the nearest minimum points in the parameter space, so as to optimise the parameters in a machine learning model with multiple parameters.

The quizzes provide a few example problems for us to work on, but as mentioned earlier, they are of the more basic variety; it is quite unlikely that undergraduate courses have examples that are this straightforward. However, I feel that this is a good thing, given that their aim is only to allow us to get a feel for multivariable calculus without bogging us down with needless complexity.

The overall aim of the course is to build intuition, which I think it accomplishes.

However, compared to the previous course in this specialization, it is harder to draw the links between the material that is covered in one week as compared to the next. It is harder to see how they are related, and how the material for each week fits into the overall picture. This was not the case in the previous course. The concepts from the previous weeks would be seemlessly integrated into those from the current week. There seems to be an unspoken expectation that the course participant should refer to external resources to fill in the blanks, and find the coherence within the material by themselves. I feel that the course instructors can do better at integrating the concepts taught across the weeks, so that it does not feel quite so fragmented.

By Rinat T

•Aug 1, 2018

the part about neural networks needs improvement (some more examples of simple networks, the explanation of the emergence of the sigmoid function). exercises on partial derivatives need to be focused more on various aspects of partial differentiation rather than on taking partial derivatives of some complicated functions. I felt like there was too much of the latter which is not very efficient because the idea of partial differentiation is easy to master but not always its applications. just taking partial derivatives of some sophisticated functions (be it for the sake of Jacobian or Hessian calculation) turns into just doing lots of algebra the idea behind which has been long understood. so while some currently existing exercises on partial differentiation, Jacobian and Hessian should be retained, about 50 percent or so of them should be replaced with exercises which are not heavy on algebra but rather demonstrate different ways and/or applications in which partial differentiation is used. otherwise all good.

By Yaroslav K

•Apr 8, 2020

1) Totally British English with a bunch of very rare-used words and phrases globally. 2) The pace of the course is just not suitable for me. If you don't have strong math or engineer background you will need to search for the explanations somewhere else (khan academy - a great resource, etc.). Closer to the end of the course I stopped having a full understanding of what's going on and why. So I could calculate things, but I don't feel that I will able to that in 1-2 week because I didn't have a time and opportunity to strengthen gained skills. 3) Also I don't understand why instructors (especially David) don't visualize what they say like Sal or Grant are doing. They draw on the desk and on the plots and so on. Sometime it looks like you just listen to audio-book about the Math.

I will take Stanford ML course after this course and also review what I've learned here with Khan Academy resource.

By Vitor R C

•Sep 18, 2020

Another great introduction to a very hard content that is Multivariate Calculus, including derivatives, but still good enough for someone with a very little mathematic basis to understand

One critique that I have is the lack of a smooth progression between the examples used in the video with the ones presented in the quizzes, sometimes the questions in the quiz are an entirely different order of difficulty than the ones in the videos.

Another critique is the seemly dive in quality in the content of the videos in the last two "weeks" of the course, you can see that very well because theses weeks have at most 20 min worth of videos each, even though it's supposed to be done during an entire week, and the content is very shallow, quick and hard to understand.

By Nikos B

•Sep 23, 2023

Very good MOOC course! I think in the last two weeks with the concepts of optimization and linear regression should be given a short introduction to statistical requirements for a better understanding of the best fit for people without background in regression. I think the course has a decreasing teaching video quality after week 4 because the concepts are considered to be known from previous courses and studies. Likewise, I am not so confident about linear regression concept in this course. Overall, I think it is one of the best mathematical introduction I have seen about nonlinear optimization regarding gradients, the steepest gradient algorithm and concepts from differential calculus as well as the backpropagation algorithm for neural networks.

By JustsaiyanHS

•Jan 4, 2021

A lot of the material Sam taught (first 4 weeks) felt very intuitive, his metaphors before introducing the concept and the following extrapolations into multivariate calc were easy to grasp. David teaches the last 2 weeks and I could no longer use the course as a starting point. I felt he overestimated prior knowledge of students and paced the lectures a bit too fast, often introducing 3-4 concepts in a short tangent.

That being said, I made it through with relative ease. The examples and labs were great and I used 3blue1brown / Khan Academy / calcworkshop (just the free lectures) to supplement my learning. I do have a good prior amount of CS, but most takes should feel comfortable enough in the jupyter environment.

By Jack C

•May 31, 2020

Great course! It was a pleasure to learn Multivariate Calculus, and Sam Cooper was great! I was even able to understand Neural Networks, which I had always found confusing! However, surprisingly, the final two weeks taught by David Dye about Optimisation and Regression were not taught well. I did not understand how to use them in practice, and the main reason why is because of Gradient Descent, an important algorithm, was not explained very well. The reason why this was so surprising is that David Dye was amazing in Linear Algebra, and I understood everything very well. Thank you Imperial College London for this great course, and I hope you edit it to explain Gradient Descent better.

By Deborah S

•Mar 25, 2021

It was really fun to actually see back propagation and gradient descent actually working - thanks for a really fun experience. I'm sure some thought went into that. I DO regret that the Jupyter Notebooks aren't made available for download. I like to work in my local environment; spent a lot of time copying code etc.. I usually was able to get this working AOK. But not for the backpropagation network from lesson 3 (the "learn to draw a heart" exercise).

Any chance you could send the notebook for that one lesson???

Anyway thanks. I'm sure it's not easy designing courses where the audience is "assume knows nothing" coupled with "must teach something substantial". NOT easy!!

By Matteo L

•Apr 20, 2020

This course is a great refresher for someone who has already studied these topics previously. The topics were very well illustrated and the objective of getting a good intuition of the math is achieved in my opinion.

I thought the examples like the neural network and the sandpits were great. That being said, I'd have liked to go a little bit deeper on the subject of optimization.

In general, I do feel that it would have been nice to have more practice on the topics (e.g. linear approximation and its use were not covered very thoroughly in my opinion). Also, the notebook assignments are far too easy and therefore don't add enough to the learning experience.

By Ronny A

•Jun 27, 2018

Course is pretty good. I like how well thought out the assignments are and the use of visualizations, even in the assignments, to enrich intuitive understanding. There were a couple of instances where the content wasn't clear and I referenced Khan Academy to clarify things for myself. The reason I give this course a 4-start rather than a 5-star is that it seems the teachers or else TAs were not responsive. Specifically, myself and another person had posted in the discussion forum how it seemed one of the slides had a typo in the Jacobian contour plot. There was no official response to this.