About this course: This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future.
Who is this class for: This class is for people who would like to learn more about machine learning techniques, but don’t currently have the fundamental mathematics in place to go into much detail. This course will include some exercises that require you to work with code. If you've not had much experience with code before DON'T PANIC, we will give you lots of guidance as you go.
Created by: Imperial College London
Taught by: Samuel J. Cooper, Lecturer
Dyson School of Design Engineering
Taught by: David Dye, Professor of Metallurgy
Department of Materials
Taught by: A. Freddie Page, Strategic Teaching Fellow
Dyson School of Design Engineering
| Basic Info | Course 2 of 3 in the Mathematics for Machine Learning Specialization |
| Level | Beginner |
| Commitment | 6 weeks of study, 2-5 hours/week |
| Language | English |
| How To Pass | Pass all graded assignments to complete the course. |
| User Ratings |
Syllabus
WEEK 1
What is calculus?
Understanding calculus is central to understanding machine learning! You can think of calculus as simply a set of tools for analysing the relationship between functions and their inputs. Typically, in machine learning, we are trying to find the inputs which enable a function to best match the data.
We start this module from the basics, by recalling what a function is and where we might encounter one. Following this, we talk about the how, when sketching a function on a graph, the slope describes the rate of change off the output with respect to an input. Using this visual intuition we next derive a robust mathematical definition of a derivative, which we then use to differentiate some interesting functions. Finally, by studying a few examples, we develop four handy time saving rules that enable us to speed up differentiation for many common scenarios.
10 videos, 4 readings, 5 practice quizzes
- Reading: About Imperial College & the team
- Reading: How to be successful in this course
- Reading: Grading Policy
- Reading: Additional Readings & Helpful References
- Discussion Prompt: Nice to meet you!
- Ungraded Plugin: Survey
- Video: Welcome to Module 1!
- Video: Functions
- Practice Quiz: Matching functions visually
- Video: Rise Over Run
- Practice Quiz: Matching the graph of a function to the graph of its derivative
- Video: Definition of a derivative
- Video: Differentiation examples & special cases
- Practice Quiz: Let's differentiate some functions
- Video: Product rule
- Practice Quiz: Practicing the product rule
- Video: Chain rule
- Practice Quiz: Practicing the chain rule
- Video: Taming a beast
- Video: See you next module!
Graded: Unleashing the toolbox
WEEK 2
Multivariate calculus
Building on the foundations of the previous module, we now generalise our calculus tools to handle multivariable systems. This means we can take a function with multiple inputs and determine the influence of each of them separately. It would not be unusual for a machine learning method to require the analysis of a function with thousands of inputs, so we will also introduce the linear algebra structures necessary for storing the results of our multivariate calculus analysis in an orderly fashion.
9 videos, 4 practice quizzes
- Video: Variables, constants & context
- Video: Differentiate with respect to anything
- Practice Quiz: Practicing partial differentiation
- Video: The Jacobian
- Practice Quiz: Calculating the Jacobian
- Video: Jacobian applied
- Practice Quiz: Bigger Jacobians!
- Video: The Sandpit
- Notebook: The Sandpit
- Notebook: The Sandpit - Part 2
- Video: The Hessian
- Practice Quiz: Calculating Hessians
- Video: Reality is hard
- Video: See you next module!
Graded: Assessment: Jacobians and Hessians
WEEK 3
Multivariate chain rule and its applications
Having seen that multivariate calculus is really no more complicated than the univariate case, we now focus on applications of the chain rule. Neural networks are one of the most popular and successful conceptual structures in machine learning. They are build up from a connected web of neurons and inspired by the structure of biological brains. The behaviour of each neuron is influenced by a set of control parameters, each of which needs to be optimised to best fit the data. The multivariate chain rule can be used to calculate the influence of each parameter of the networks, allow them to be updated during training.
6 videos, 3 practice quizzes
- Video: Multivariate chain rule
- Video: More multivariate chain rule
- Practice Quiz: Multivariate chain rule exercise
- Video: Simple neural networks
- Practice Quiz: Simple Artificial Neural Networks
- Video: More simple neural networks
- Practice Quiz: Training Neural Networks
- Notebook: Backpropagation
- Discussion Prompt: I ❤️ backpropagation
- Video: See you next module!
Graded: Backpropagation
WEEK 4
Taylor series and linearisation
The Taylor series is a method for re-expressing functions as polynomial series. This approach is the rational behind the use of simple linear approximations to complicated functions. In this module, we will derive the formal expression for the univariate Taylor series and discuss some important consequences of this result relevant to machine learning. Finally, we will discuss the multivariate case and see how the Jacobian and the Hessian come in to play.
9 videos, 4 practice quizzes
- Video: Building approximate functions
- Video: Power series
- Practice Quiz: Matching functions and approximations
- Ungraded Plugin: Visualising Taylor Series
- Video: Power series derivation
- Video: Power series details
- Practice Quiz: Applying the Taylor series
- Video: Examples
- Video: Linearisation
- Practice Quiz: Taylor series - Special cases
- Video: Multivariate Taylor
- Practice Quiz: 2D Taylor series
- Video: See you next module!
Graded: Taylor Series Assessment
WEEK 5
Intro to optimisation
If we want to find the minimum and maximum points of a function then we can use multivariate calculus to do this, say to optimise the parameters (the space) of a function to fit some data. First we’ll do this in one dimension and use the gradient to give us estimates of where the zero points of that function are, and then iterate in the Newton-Raphson method. Then we’ll extend the idea to multiple dimensions by finding the gradient vector, Grad, which is the vector of the Jacobian. This will then let us find our way to the minima and maxima in what is called the gradient descent method. We’ll then take a moment to use Grad to find the minima and maxima along a constraint in the space, which is the Lagrange multipliers method.
4 videos, 2 practice quizzes
- Practice Quiz: Newton-Raphson in one dimension
- Video: Gradient Descent
- Notebook: Gradient descent in a sandpit
- Discussion Prompt: Steepest strategies
- Video: Constrained optimisation
- Practice Quiz: Lagrange multipliers
- Video: See you next module!
Graded: Checking Newton-Raphson
Graded: Optimisation scenarios
WEEK 6
Regression
In order to optimise the fitting parameters of a fitting function to the best fit for some data, we need a way to define how good our fit is. This goodness of fit is called chi-squared, which we’ll first apply to fitting a straight line - linear regression. Then we’ll look at how to optimise our fitting function using chi-squared in the general case using the gradient descent method. Finally, we’ll look at how to do this easily in Python in just a few lines of code, which will wrap up the course.
4 videos, 1 reading, 2 practice quizzes
- Practice Quiz: Linear regression
- Video: General non linear least squares
- Practice Quiz: Fitting a non-linear function
- Video: Doing least squares regression analysis in practice
- Notebook: Fitting the distribution of heights data
- Video: Wrap up of this course
- Reading: Did you like the course? Let us know!
- Ungraded Plugin: Post-Course Survey
Graded: Fitting the distribution of height data
FAQs
How It Works
Coursework
Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.
Help from Your Peers
Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.
Certificates
Earn official recognition for your work, and share your success with friends, colleagues, and employers.
Creators
Imperial College London
Imperial College London is a world top ten university with an international reputation for excellence in science, engineering, medicine and business. located in the heart of London. Imperial is a multidisciplinary space for education, research, translation and commercialisation, harnessing science and innovation to tackle global challenges.
Imperial students benefit from a world-leading, inclusive educational experience, rooted in the College’s world-leading research. Our online courses are designed to promote interactivity, learning and the development of core skills, through the use of cutting-edge digital technology.
Pricing
| Purchase Course | |
|---|---|
| Access to course materials | Available |
| Access to graded materials | Available |
| Receive a final grade | Available |
| Earn a shareable Course Certificate | Available |
Ratings and Reviews
a very good explanation of the required calculus basics for machine learning. moreover, it opens the way for the wide optimization world.
excellent course!
Really good course, I learned a lot from here. This course makes me feel that math is neat and cool.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
Excellent course! It helps understand to take the sandpit as an example for learning Jacobian, Hessian and steepest algorithm stuff. More than boring math formulas.