About this Course
Learn about artificial neural networks and how they're being used for machine learning, as applied to speech and object recognition, image segmentation, modeling language and human motion, etc. We'll emphasize both the basic algorithms and the practical tricks needed to get them to work well. This course contains the same content presented on Coursera beginning in 2013. It is not a continuation or update of the original course. It has been adapted for the new platform. Please be advised that the course is suited for an intermediate level learner - comfortable with calculus and with experience programming (Python).
Globe

100% online course

Start instantly and learn at your own schedule.
Clock

Approx. 45 hours to complete

Suggested: 5 hours/week
Comment Dots

English

Subtitles: English

Skills you will gain

Artificial Neural NetworkMachine LearningBayesian NetworkMathematical Optimization
Globe

100% online course

Start instantly and learn at your own schedule.
Clock

Approx. 45 hours to complete

Suggested: 5 hours/week
Comment Dots

English

Subtitles: English

Syllabus - What you will learn from this course

1

Section
Clock
2 hours to complete

Introduction

Introduction to the course - machine learning and neural nets...
Reading
5 videos (Total 43 min), 8 readings, 1 quiz
Video5 videos
What are neural networks? [8 min]8m
Some simple models of neurons [8 min]8m
A simple example of learning [6 min]5m
Three types of learning [8 min]7m
Reading8 readings
Syllabus and Course Logistics10m
Lecture Slides (and resources)10m
Setting Up Your Programming Assignment Environment10m
Installing Octave on Windows10m
Installing Octave on Mac OS X (10.10 Yosemite and 10.9 Mavericks)10m
Installing Octave on Mac OS X (10.8 Mountain Lion and Earlier)10m
Installing Octave on GNU/Linux10m
More Octave10m
Quiz1 practice exercises
Lecture 1 Quiz12m

2

Section
Clock
1 hour to complete

The Perceptron learning procedure

An overview of the main types of neural network architecture ...
Reading
5 videos (Total 42 min), 1 reading, 1 quiz
Video5 videos
Perceptrons: The first generation of neural networks [8 min]8m
A geometrical view of perceptrons [6 min]6m
Why the learning works [5 min]5m
What perceptrons can't do [15 min]14m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 2 Quiz16m

3

Section
Clock
1 hour to complete

The backpropagation learning proccedure

Learning the weights of a linear neuron ...
Reading
5 videos (Total 43 min), 2 readings, 2 quizzes
Video5 videos
The error surface for a linear neuron [5 min]5m
Learning the weights of a logistic output neuron [4 min]3m
The backpropagation algorithm [12 min]11m
Using the derivatives computed by backpropagation [10 min]9m
Reading2 readings
Lecture Slides (and resources)10m
Forward Propagation in Neural Networks10m
Quiz2 practice exercises
Lecture 3 Quiz12m
Programming Assignment 1: The perceptron learning algorithm.12m

4

Section
Clock
1 hour to complete

Learning feature vectors for words

Learning to predict the next word...
Reading
5 videos (Total 44 min), 1 reading, 1 quiz
Video5 videos
A brief diversion into cognitive science [4 min]4m
Another diversion: The softmax output function [7 min]7m
Neuro-probabilistic language models [8 min]7m
Ways to deal with the large number of possible outputs [15 min]12m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 4 Quiz14m

5

Section
Clock
2 hours to complete

Object recognition with neural nets

In this module we look at why object recognition is difficult. ...
Reading
4 videos (Total 44 min), 1 reading, 2 quizzes
Video4 videos
Achieving viewpoint invariance [6 min]5m
Convolutional nets for digit recognition [16 min]16m
Convolutional nets for object recognition [17min]17m
Reading1 readings
Lecture Slides (and resources)10m
Quiz2 practice exercises
Lecture 5 Quiz12m
Programming Assignment 2: Learning Word Representations.26m

6

Section
Clock
1 hour to complete

Optimization: How to make the learning go faster

We delve into mini-batch gradient descent as well as discuss adaptive learning rates....
Reading
5 videos (Total 48 min), 1 reading, 1 quiz
Video5 videos
A bag of tricks for mini-batch gradient descent13m
The momentum method8m
Adaptive learning rates for each connection5m
Rmsprop: Divide the gradient by a running average of its recent magnitude11m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 6 Quiz10m

7

Section
Clock
1 hour to complete

Recurrent neural networks

This module explores training recurrent neural networks...
Reading
5 videos (Total 47 min), 1 reading, 1 quiz
Video5 videos
Training RNNs with back propagation6m
A toy example of training an RNN6m
Why it is difficult to train an RNN7m
Long-term Short-term-memory9m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 7 Quiz12m

8

Section
Clock
1 hour to complete

More recurrent neural networks

We continue our look at recurrent neural networks...
Reading
3 videos (Total 37 min), 1 reading, 1 quiz
Video3 videos
Learning to predict the next character using HF [12 mins]12m
Echo State Networks [9 min]9m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 8 Quiz14m

9

Section
Clock
2 hours to complete

Ways to make neural networks generalize better

We discuss strategies to make neural networks generalize better...
Reading
6 videos (Total 51 min), 1 reading, 2 quizzes
Video6 videos
Limiting the size of the weights [6 min]6m
Using noise as a regularizer [7 min]7m
Introduction to the full Bayesian approach [12 min]10m
The Bayesian interpretation of weight decay [11 min]10m
MacKay's quick and dirty method of setting weight costs [4 min]3m
Reading1 readings
Lecture Slides (and resources)10m
Quiz2 practice exercises
Lecture 9 Quiz12m
Programming assignment 3: Optimization and generalization22m

10

Section
Clock
1 hour to complete

Combining multiple neural networks to improve generalization

This module we look at why it helps to combine multiple neural networks to improve generalization...
Reading
5 videos (Total 49 min), 1 reading, 1 quiz
Video5 videos
Mixtures of Experts [13 min]13m
The idea of full Bayesian learning [7 min]7m
Making full Bayesian learning practical [7 min]6m
Dropout [9 min]8m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 10 Quiz12m

11

Section
Clock
1 hour to complete

Hopfield nets and Boltzmann machines

...
Reading
5 videos (Total 56 min), 1 reading, 1 quiz
Video5 videos
Dealing with spurious minima [11 min]11m
Hopfield nets with hidden units [10 min]9m
Using stochastic units to improv search [11 min]10m
How a Boltzmann machine models data [12 min]11m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 11 Quiz10m

12

Section
Clock
1 hour to complete

Restricted Boltzmann machines (RBMs)

This module deals with Boltzmann machine learning ...
Reading
5 videos (Total 53 min), 1 reading, 1 quiz
Video5 videos
OPTIONAL VIDEO: More efficient ways to get the statistics [15 mins]14m
Restricted Boltzmann Machines [11 min]10m
An example of RBM learning [7 mins]7m
RBMs for collaborative filtering [8 mins]8m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 12 Quiz14m

13

Section
Clock
1 hour to complete

Stacking RBMs to make Deep Belief Nets

...
Reading
3 videos (Total 36 min), 1 reading, 2 quizzes
Video3 videos
Belief Nets [13 min]12m
The wake-sleep algorithm [13 min]13m
Reading1 readings
Lecture Slides (and resources)10m
Quiz2 practice exercises
Programming Assignment 4: Restricted Boltzmann Machines22m
Lecture 13 Quiz14m

14

Section
Clock
1 hour to complete

Deep neural nets with generative pre-training

...
Reading
5 videos (Total 63 min), 1 reading, 1 quiz
Video5 videos
Discriminative learning for DBNs [9 mins]9m
What happens during discriminative fine-tuning? [8 mins]8m
Modeling real-valued data with an RBM [10 mins]9m
OPTIONAL VIDEO: RBMs are infinite sigmoid belief nets [17 mins]17m
Reading1 readings
Lecture Slides (and resources)10m
Quiz1 practice exercises
Lecture 14 Quiz10m

15

Section
Clock
2 hours to complete

Modeling hierarchical structure with neural nets

...
Reading
6 videos (Total 46 min), 1 reading, 2 quizzes
Video6 videos
Deep auto encoders [4 mins]4m
Deep auto encoders for document retrieval [8 mins]8m
Semantic Hashing [9 mins]8m
Learning binary codes for image retrieval [9 mins]9m
Shallow autoencoders for pre-training [7 mins]7m
Reading1 readings
Lecture Slides (and resources)10m
Quiz2 practice exercises
Lecture 15 Quiz14m
Final Exam36m

16

Section
Clock
1 hour to complete

Recent applications of deep neural nets

...
Reading
3 videos (Total 32 min)
Video3 videos
OPTIONAL: Hierarchical Coordinate Frames [10 mins]9m
OPTIONAL: Bayesian optimization of hyper-parameters [13 min]13m
4.6
Direction Signs

28%

started a new career after completing these courses
Briefcase

83%

got a tangible career benefit from this course

Top Reviews

By NKJul 23rd 2017

Excellent content! Great to learn from a pioneer of the field.\n\nThe material is hard to digest, sometimes. Less text and a pointer would have helped.\n\nAnyway, great course. Thank you Prof. Hinton

By NSAug 13th 2017

Although It was way too tough for me, but you have to agree that you learn a lot throughout the course.\n\nI'll definitely pursue some other courses related to Deep Learning here.\n\nThanks Coursera.

Instructor

About University of Toronto

Established in 1827, the University of Toronto has one of the strongest research and teaching faculties in North America, presenting top students at all levels with an intellectual environment unmatched in depth and breadth on any other Canadian campus. ...

Frequently Asked Questions

  • Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.

  • If you pay for this course, you will have access to all of the features and content you need to earn a Course Certificate. If you complete the course successfully, your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. Note that the Course Certificate does not represent official academic credit from the partner institution offering the course.

  • Yes! Coursera provides financial aid to learners who would like to complete a course but cannot afford the course fee. To apply for aid, select "Learn more and apply" in the Financial Aid section below the "Enroll" button. You'll be prompted to complete a simple application; no other paperwork is required.

More questions? Visit the Learner Help Center