Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

This course is part of the Deep Learning Specialization

4.9
stars
58,561 ratings

Andrew Ng +2 more instructors
Top Instructors

401,029 already enrolled

Offered By

Deep Learning SpecializationDeepLearning.AI

About this Course

582,100 recent views

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. 
By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow.

The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI.

Learner Career Outcomes

28%

started a new career after completing these courses

28%

got a tangible career benefit from this course

11%

got a pay increase or promotion

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 2 of 5 in the

Deep Learning Specialization

Intermediate Level

Intermediate Python skills: basic programming, understanding of for loops, if/else statements, data structures
A basic grasp of linear algebra & ML

Approx. 22 hours to complete

English

Subtitles: Chinese (Traditional), Arabic, French, Portuguese (European), Chinese (Simplified), Italian, Portuguese (Brazilian), Vietnamese, Korean, German, Russian, Turkish, English, Spanish

Skills you will gain

TensorflowDeep LearningMathematical Optimizationhyperparameter tuning

Learner Career Outcomes

28%

started a new career after completing these courses

28%

got a tangible career benefit from this course

11%

got a pay increase or promotion

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Course 2 of 5 in the

Deep Learning Specialization

Intermediate Level

Intermediate Python skills: basic programming, understanding of for loops, if/else statements, data structures
A basic grasp of linear algebra & ML

Approx. 22 hours to complete

English

Subtitles: Chinese (Traditional), Arabic, French, Portuguese (European), Chinese (Simplified), Italian, Portuguese (Brazilian), Vietnamese, Korean, German, Russian, Turkish, English, Spanish

Instructors

Instructor rating

4.92/5 (3,609 Ratings)

Offered by

DeepLearning.AI

DeepLearning.AI is an education technology company that develops a global community of AI talent.

Syllabus - What you will learn from this course

Content Rating96%(60,952 ratings)

Week

Week 1

12 hours to complete

Practical Aspects of Deep Learning

Discover and experiment with a variety of different initialization methods, apply L2 regularization and dropout to avoid model overfitting, then apply gradient checking to identify errors in a fraud detection model.

12 hours to complete

15 videos (Total 131 min), 8 readings, 4 quizzes

15 videos

Train / Dev / Test sets12m

Bias / Variance8m

Basic Recipe for Machine Learning6m

Regularization9m

Why Regularization Reduces Overfitting?7m

Dropout Regularization9m

Understanding Dropout7m

Other Regularization Methods8m

Normalizing Inputs5m

Vanishing / Exploding Gradients6m

Weight Initialization for Deep Networks6m

Numerical Approximation of Gradients6m

Gradient Checking6m

Gradient Checking Implementation Notes5m

Yoshua Bengio Interview25m

8 readings

Connect with your Mentors and Fellow Learners on Discourse!10m

Guide to Discourse10m

Clarification about Upcoming Regularization Video1m

Clarification about Upcoming Understanding Dropout Video1m

Clarification about Upcoming Normalizing Inputs Video1m

Lectures in PDF1m

How to Download your Notebook

How to Refresh your Workspace5m

1 practice exercise

Practical aspects of Deep Learning20m

Week

Week 2

5 hours to complete

Optimization Algorithms

Develop your deep learning toolbox by adding more advanced optimizations, random minibatching, and learning rate decay scheduling to speed up your models.

5 hours to complete

11 videos (Total 92 min), 3 readings, 2 quizzes

11 videos

Mini-batch Gradient Descent11m

Understanding Mini-batch Gradient Descent11m

Exponentially Weighted Averages5m

Understanding Exponentially Weighted Averages9m

Bias Correction in Exponentially Weighted Averages4m

Gradient Descent with Momentum9m

RMSprop7m

Adam Optimization Algorithm7m

Learning Rate Decay6m

The Problem of Local Optima5m

Yuanqing Lin Interview13m

3 readings

Clarification about Upcoming Adam Optimization Video1m

Clarification about Learning Rate Decay Video1m

Lectures in PDF1m

1 practice exercise

Optimization Algorithms20m

Week

Week 3

5 hours to complete

Hyperparameter Tuning, Batch Normalization and Programming Frameworks

Explore TensorFlow, a deep learning framework that allows you to build neural networks quickly and easily, then train a neural network on a TensorFlow dataset.

5 hours to complete

11 videos (Total 103 min), 5 readings, 2 quizzes

11 videos

Tuning Process7m

Using an Appropriate Scale to pick Hyperparameters8m

Hyperparameters Tuning in Practice: Pandas vs. Caviar6m

Normalizing Activations in a Network8m

Fitting Batch Norm into a Neural Network12m

Why does Batch Norm work?11m

Batch Norm at Test Time5m

Softmax Regression11m

Training a Softmax Classifier10m

Deep Learning Frameworks4m

TensorFlow15m

5 readings

Clarifications about Upcoming Softmax Video1m

(Optional) Learn about Gradient Tape and More1m

Lectures in PDF1m

References10m

Acknowledgments10m

1 practice exercise

Hyperparameter tuning, Batch Normalization, Programming Frameworks20m

Reviews

4.9

6745 reviews

5 stars
88.33%
4 stars
10.53%
3 stars
0.96%
2 stars
0.10%
1 star
0.05%

TOP REVIEWS FROM IMPROVING DEEP NEURAL NETWORKS: HYPERPARAMETER TUNING, REGULARIZATION AND OPTIMIZATION

by YLFeb 20, 2018

very useful course, especially the last tensorflow assignment. the only reason i gave 4 stars is due to the lack of practice on batchnorm, which i believe is one of the most usefule techniques lately.

by NAJan 13, 2020

After completion of this course I know which values to look at if my ML model is not performing up to the task. It is a detailed but not too complicated course to understand the parameters used by ML.

by JSApr 4, 2021

Fantastic course and although it guides you through the course (and may feel less challenging to some) it provides all the building blocks for you to latter apply them to your own interesting project.

by AMOct 8, 2019

I really enjoyed this course. Many details are given here that are crucial to gain experience and tips on things that looks easy at first sight but are important for a faster ML project implementation

View all reviews

About the Deep Learning Specialization

The Deep Learning Specialization is a foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology.

In this Specialization, you will build and train neural network architectures such as Convolutional Neural Networks, Recurrent Neural Networks, LSTMs, Transformers, and learn how to make them better with strategies such as Dropout, BatchNorm, Xavier/He initialization, and more. Get ready to master theoretical concepts and their industry applications using Python and TensorFlow and tackle real-world cases such as speech recognition, music synthesis, chatbots, machine translation, natural language processing, and more. AI is transforming many industries. The Deep Learning Specialization provides a pathway for you to take the definitive step in the world of AI by helping you gain the knowledge and skills to level up your career. Along the way, you will also get career advice from deep learning experts from industry and academia.

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Is financial aid available?
Yes, Coursera provides financial aid to learners who cannot afford the fee. Apply for it by clicking on the Financial Aid link beneath the "Enroll" button on the left. You'll be prompted to complete an application and will be notified if you are approved. You'll need to complete this step for each course in the Specialization, including the Capstone Project. Learn more.
Will I earn university credit for completing the Course?
This Course doesn't carry university credit, but some universities may choose to accept Course Certificates for credit. Check with your institution to learn more. Online Degrees and Mastertrack™ Certificates on Coursera provide the opportunity to earn university credit.

More questions? Visit the Learner Help Center.

Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

About this Course

Learner Career Outcomes

28%

28%

11%

Skills you will gain

Learner Career Outcomes

28%

28%

11%

Instructors

Andrew Ng
Top Instructor

Kian Katanforoosh
Top Instructor

Younes Bensouda Mourri
Top Instructor

Offered by

DeepLearning.AI

Syllabus - What you will learn from this course

Week 1

Practical Aspects of Deep Learning

Week 2

Optimization Algorithms

Week 3

Hyperparameter Tuning, Batch Normalization and Programming Frameworks

Reviews

TOP REVIEWS FROM IMPROVING DEEP NEURAL NETWORKS: HYPERPARAMETER TUNING, REGULARIZATION AND OPTIMIZATION

About the Deep Learning Specialization

Frequently Asked Questions

Start or advance your career

Browse popular topics

Upskill your team

Earn a degree or certificate online

Coursera

Community

More

Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

About this Course

Learner Career Outcomes

28%

28%

11%

Skills you will gain

Learner Career Outcomes

28%

28%

11%

Instructors

Andrew NgTop Instructor

Kian KatanforooshTop Instructor

Younes Bensouda MourriTop Instructor

Offered by

DeepLearning.AI

Syllabus - What you will learn from this course

Week 1

Practical Aspects of Deep Learning

Week 2

Optimization Algorithms

Week 3

Hyperparameter Tuning, Batch Normalization and Programming Frameworks

Reviews

TOP REVIEWS FROM IMPROVING DEEP NEURAL NETWORKS: HYPERPARAMETER TUNING, REGULARIZATION AND OPTIMIZATION

About the Deep Learning Specialization

Frequently Asked Questions

Coursera Footer

Start or advance your career

Browse popular topics

Upskill your team

Earn a degree or certificate online

Coursera

Community

More

Andrew Ng
Top Instructor

Kian Katanforoosh
Top Instructor

Younes Bensouda Mourri
Top Instructor