Back to Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

4.9

stars

42,913 ratings

•

4,599 reviews

This course will teach you the "magic" of getting deep learning to work well. Rather than the deep learning process being a black box, you will understand what drives performance, and be able to more systematically get good results. You will also learn TensorFlow.
After 3 weeks, you will:
- Understand industry best-practices for building deep learning applications.
- Be able to effectively use the common neural network "tricks", including initialization, L2 and dropout regularization, Batch normalization, gradient checking,
- Be able to implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence.
- Understand new best-practices for the deep learning era of how to set up train/dev/test sets and analyze bias/variance
- Be able to implement a neural network in TensorFlow.
This is the second course of the Deep Learning Specialization....

Dec 06, 2019

I enjoyed it, it is really helpful, id like to have the oportunity to implement all these deeply in a real example.\n\nthe only thing i didn't have completely clear is the barch norm, it is so confuse

Dec 24, 2017

Exceptional Course, the Hyper parameters explanations are excellent every tip and advice provided help me so much to build better models, I also really liked the introduction of Tensor Flow\n\nThanks.

Filter by:

By Jairo J P H

•Feb 01, 2020

El curso es muy bueno, particularmente estoy muy agradecido con COURSERA, por darme la oportunidad de hacer los cinco cursos de la Especialización en Deep Learning con ayuda economica y permitirme tener acceso a este tipo de capacitacion y certificacion. Muchas Gracias…!

The course is very good, particularly I am very grateful to COURSERA, for giving me the opportunity to do the five courses of the Deep Learning Specialization with financial aid and allowing me to have access to this type of training and certification. Thank you very much!

By David C

•Dec 29, 2017

This is a logical continuation of the previous course. The 3-week topics were excellently chosen. Andrew did a great job of delivering the lectures. The programming assignments really reinforced my understanding. In particular, essential knowledge and equations from video lectures were reiterated in the programming assignments for review and ease of reference. The amount of work was reasonable, and the level of challenge was appropriate. I especially appreciate the instructional team for making this course open to the public.

By Alessandro T

•Jan 22, 2018

A right balance between theory (you are required to code know the models and code them from scratch) and practice (you get an overview of the frameworks available out there to put your code into production quickly and efficiently; and time is spent on practical aspects of the training phase).

A small "criticism": in the notebook, more than programming you just have to fill a template where a good part of the algorithm is already drafted for you. It is too much, students should be left scratching their heads a bit longer :)

By Heshmat S

•Dec 27, 2017

This is the 2nd course from Andrew Ng in the "deep learning specialization". Having introduced the building blocks of deep neural networks, in this course Andrew teaches more advanced and practical concepts - like: regularization, advanced optimization techniques, batch-normalization, etc - that can significantly improve the implementation of the models we build.

Also, in this course we get to learn TensorFlow, a widely used and wonderful deep learning framework.

I highly recommend this course.

Thank you Andrew & Co. :-)

By Beng C C

•Dec 31, 2018

Great course! But I am not too sure why this should be placed in number 2, as I feel that topics such as tuning hyperparameters do not resonate well with someone who is not working professionally or is not very experienced in this field. However, still a great course as I will revisit this course when I gain more experience. I also like the last exercise on Tensorflow as there is a lack of courses on Tensorflow on the Internet, so the last assignment on Tensorflow is the most useful which I have found in the course.

By Hassan S

•Apr 03, 2018

Andrew Ng and the teaching assistants' team of this class are obviously very very determined not to leave any single major subject in deep learning without coverage. I have been using deep learning for the past couple years, but I have to say by completing the second course of this specialization, they helped me deepen my understanding, overcome fear of implementing math and equations line by line, fix my intuitions about deep learning, and most importantly erase all the superstitions! Bravo and excellent job.

By Glenn B

•May 31, 2018

Course material was great, however the use of Tensorflow in the exercises requires more background than provided in the short tutorial.

I get the dynamic aspect of writing the lecture notes in the videos, however the lecture notes should be "cleaned up" in the downloadable files (i.e., typos corrected and typed up). Additionally, the notes written in the video could be written and organized more clearly (e.g., uniform directional flow across the page/screen rather than randomly fit wherever on the page.

By Svetlana L

•Oct 22, 2019

I liked that the course gradually introduces more and more complexity and concepts without making your drown. Even though existing frameworks (e.g. tensorflow) can be used so that most of the complexity is hidden it is still required to understand why one method should be used rather than the other. This course I believe addresses this (as well as first in this specialisation). I still wish there was more information on details but probably all that is needed are external links to extra material.

By Robert K

•Nov 18, 2017

Fantastic course! You can experience short, easy to understand lectures, followed by plenty of opportunities to implement covered material, and most importantly create optimized image classifiers - like cats, dogs. I liked how up until the end of the course you had to implement everything from the scratch, not just using read-made frameworks. Finally, you are introduced into frameworks, but this deep understanding stays with you. 5/5 recommendation. Bye, I gotta finish the rest of specialization.

By Ferenc F P

•Mar 08, 2018

Good course explaining the concept of hyperparameters vs. parameters, how you can tune the hyperparameters, as well as different regularization techniques. It also provides good explanation for different optimization algorithms (enhancements to stochastic gradient descent). It is a highly recommended course for those who want to understand what is happening under the hood when using a neural network framework, like tensorflow. In the last week a brief introduction to tensorflow is also provided.

By Akanksha D

•Dec 31, 2017

The course is great as I expected. It would be helpful if more mathematical background in videos or notes can be attached in each weeks. Moreover, more code could be given to us to write by ourselves to get much better intuition. Rest each of the specializations are awesome as was the first learning Andrew Ng course on Machine Learning. Thank you for providing such courses. This is a great deal for all such students who cannot afford to attend Ivy leagues due to their own reasons.

Great Work!!

By Kévin S

•Jul 31, 2018

It explain neural network from the start. After doing all the 5 courses on deeplearning, it is hard to remember normalization formula, and every details. Sometime some hyperparameter look like a little bullshit: You don't know how to do : add one hyperparameter and go for an argmax. But if it is how its work, then it is okay to learn it; Be ready to laugth and do not compare to pure methods like genetics or Bayesian programming that often work good. But every one should follow this course.

By GEORGE A

•Mar 05, 2019

Pretty solid class, learned a lot of basic concepts. The class won't go into a lot of mathematical details about the algorithms however, there is enough intuition provided in order to understand the inner workings of the algorithms and the logic behind them. The only con I have is that some of the programming exercises look outdated with the current versions of the notebook. For example, in my last exercise I couldn't make the NN with tensorflow to work properly but got 100/100 nevertheless.

By Matei I

•Feb 02, 2019

This course covers details about neural network implementations that are extremely useful in practice. In fact, after completing week 1 and learning about vanishing gradients, I was finally able to debug a NN implementation that I had been struggling with. I'm also grateful for the introduction to Tensorflow. As with the previous course in this specialization, expect to be spoon-fed during the programming assignments. The course would be better if it let you think more during assignments.

By 江小鱼

•Feb 12, 2019

This time , I finished Regularzation, I think this is a interesting experience, for you can implement your alg step by step, I get some magic(not black magic) alg, like RMS, momentum and Adam. At last, the most fascinating is to construct Tensorflow, just like a pipeline, step by step , and every step was made by only one line, from forward (without backward) to the model, Tensorflow is really black magic.

(I have to say Tensorflow is a bit difficult, forgive my poor English, thanks )

By Nathan Y

•Oct 16, 2017

Neural networks are not new. What we learned in this course is some of the critical implementation details/tricks from the past decades of making them work in practice. Going beyond gradient descent, types of regularization, hyperparameter searching we get to a set of robust tools that quickly find good solutions in extremely high dimensional spaces. As Professor Ng says, our understanding of optimization rules of thumb in low dimensional spaces doesn't carry over to deep learning.

By José A

•Oct 31, 2017

Seamlessly continues the previous course. If you know the basic structures of Neural Networks, how to initialize weights. Sigmoid, Tangenth, activations, and so forth, this will help you understand terms such as L2 regularization, gradient descent with momentum, RMSProp, Adam, Exponentially weighted averages, and many others.

Don't let the 3 weeks set you off. It has a lot of micro-content material that works on top of the previous work. Thanks to all the mentors for this great course.

By Raimond L

•Aug 20, 2017

Really nice course. A lot of good information about how to prepare and divide data for training, hyperparameters optimization strategy, regularization techniques, learning algorithms, mini-batches, batch-normalization and more... Very useful information with clear explanation !!! Highly recommended course.

Very positive course, except tensorflow practical assignment, which caused some stress, because for me that framework is a bit alienating, forcing to look into manual every minute.

By P M K

•Nov 26, 2017

Hi, The course content was definitely good and it helped to understand a lot of internals quite easily. I, however have one suggestion, the Introduction to Tensor Flow looked quite fast and could have been done in a better way by giving more slides about TensorFlow and then going on to the examples. Please ensure that you correct any errors pointed by the members taking this course, so that it benefits others and avoids wasting of time and reduces frustration at times.

Regards, PMK

By Sachita N

•Jun 18, 2018

Professor Ng explains the most complicated concepts in the most intuitive fashion I have ever seen. The explanations are simple, straightforward and they encompass so many perspectives and alternatives to doing things. The exercises are immensely educational - they strike a great balance between guiding the student and letting them figure stuff out on their own. This is a great specialisation and I would whole-heartedly recommend it for anyone wanting to start with Deep Learning

By kindalin

•Jul 31, 2019

This is the best course I have ever seen. The previous mooc class gave me some bad impressions, which is be created by some scholars for KPI. I believe that such a well-designed course will eventually replace the traditional curriculum. This is also a good hope for our students in non-brand schools.

The only downside is that the coursework instructions are too detailed as many people reflect. I can see a lot of good and hard designs in it, but I hope it can have a better form.

By Joppe G

•Aug 13, 2017

This course is simply brilliant. You start with implementing the low-level functions that make up a deep learning framework. It's only in the last assignment that you explore TensorFlow. At that point, you have a full understanding of what the API encapsulates.

This really gives you confidence in your capability to get started with your own projects, knowing that you can come back at any time to brush up on some of the lower-level details.

Thank you Andrew and the whole team!

By Rajeev B

•Nov 18, 2017

The assignments are very good. All the parameter update methods are explained in a very good manner. I would recommend it very strongly for anyone who is looking for an in depth understanding of why we do what we do for tuning, regularization, optimization of NN. All the implementation in the assignments is also from scratch, so, that really helps a lot. I felt this is better than Stanford CS231n course material, after all this is a whole course on this specific purpose :).

By Marcel M

•Jun 01, 2018

This course a practical way of fine tuning your model in order to improve on its performance. Rather than Deep Learning being a "so-called" black box. It turns out that Machine Learning models are not black boxes but rather there are proven techniques of not only finding out what happens in them but also to fine tune them in a systematic manner in order to improve on their results. It is an excellent course for the practical Deep Learning Engineer. Good Job and Keep It Up!

By Artem M

•Apr 22, 2018

Found a lot of interesting details about NN that I did not know. This is a much better course than the first one. IncludesTensorflow exercises, which is useful. Nevertheless, proofs are still omitted for some results like initializations. It is not hard to google, but I bet lecturers could explain them much faster than diving into science literature. Otherwise, intuitional explanations of Adam using exponential smoothing, or physics analogy of momentum are just brilliant.