This course is part of the Reinforcement Learning Specialization

4.8
stars
2,501 ratings

Martha White +1 more instructor

71,633 already enrolled

Offered By

University of Alberta

Alberta Machine Intelligence Institute

Reinforcement Learning SpecializationUniversity of Alberta

About this Course

110,588 recent views

Reinforcement Learning is a subfield of Machine Learning, but is also a general purpose formalism for automated decision-making and AI. This course introduces you to statistical learning techniques where an agent explicitly takes actions and interacts with the world. Understanding the importance and challenges of learning agents that make decisions is of vital importance today, with more and more companies interested in interactive agents and intelligent decision-making. 
This course introduces you to the fundamentals of Reinforcement Learning. When you finish this course, you will:
- Formalize problems as Markov Decision Processes 
- Understand basic exploration methods and the exploration/exploitation tradeoff
- Understand value functions, as a general-purpose tool for optimal decision-making
- Know how to implement dynamic programming as an efficient solution approach to an industrial control problem

This course teaches you the key concepts of Reinforcement Learning, underlying classic and modern algorithms in RL. After completing this course, you will be able to start using RL for real problems, where you have or can specify the MDP. 

This is the first course of the Reinforcement Learning Specialization.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Course 1 of 4 in the

Reinforcement Learning Specialization

Intermediate Level

Probabilities & Expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), implementing algorithms from pseudocode.

Approx. 15 hours to complete

English

What you will learn

Formalize problems as Markov Decision Processes
Understand basic exploration methods and the exploration / exploitation tradeoff
Understand value functions, as a general-purpose tool for optimal decision-making
Know how to implement dynamic programming as an efficient solution approach to an industrial control problem

Skills you will gain

Artificial Intelligence (AI)
Machine Learning
Reinforcement Learning
Function Approximation
Intelligent Systems

Flexible deadlines

Reset deadlines in accordance to your schedule.

Shareable Certificate

Earn a Certificate upon completion

100% online

Start instantly and learn at your own schedule.

Coursera Labs

Includes hands on learning projects.

Learn more about Coursera Labs

Course 1 of 4 in the

Reinforcement Learning Specialization

Intermediate Level

Probabilities & Expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), implementing algorithms from pseudocode.

Approx. 15 hours to complete

English

Instructors

Instructor rating

4.71/5 (714 Ratings)

Martha White

Adam White

Offered by

University of Alberta

Alberta Machine Intelligence Institute

Syllabus - What you will learn from this course

Content Rating92%(17,594 ratings)

Week

Week 1

1 hour to complete

Welcome to the Course!

Welcome to: Fundamentals of Reinforcement Learning, the first course in a four-part specialization on Reinforcement Learning brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, get a flavour of what the course has in store for you, and be given an in-depth roadmap to help make your journey through this specialization as smooth as possible.

1 hour to complete

4 videos (Total 20 min), 2 readings

4 hours to complete

An Introduction to Sequential Decision-Making

4 hours to complete

8 videos (Total 46 min), 3 readings, 2 quizzes

8 videos

Sequential Decision Making with Evaluative Feedback5m

Learning Action Values4m

Estimating Action Values Incrementally5m

What is the trade-off?7m

Optimistic Initial Values6m

Upper-Confidence Bound (UCB) Action Selection5m

Jonathan Langford: Contextual Bandits for Real World Reinforcement Learning8m

Week 1 Summary3m

3 readings

Module 1 Learning Objectives10m

Weekly Reading30m

Chapter Summary30m

1 practice exercise

Sequential Decision-Making45m

Week

Week 2

3 hours to complete

Markov Decision Processes

When you’re presented with a problem in industry, the first and most important step is to translate that problem into a Markov Decision Process (MDP). The quality of your solution depends heavily on how well you do this translation. This week, you will learn the definition of MDPs, you will understand goal-directed behavior and how this can be obtained from maximizing scalar rewards, and you will also understand the difference between episodic and continuing tasks. For this week’s graded assessment, you will create three example tasks of your own that fit into the MDP framework.

3 hours to complete

7 videos (Total 36 min), 2 readings, 2 quizzes

Week

Week 3

3 hours to complete

Value Functions & Bellman Equations

3 hours to complete

9 videos (Total 56 min), 3 readings, 2 quizzes

9 videos

Specifying Policies4m

Value Functions6m

Rich Sutton and Andy Barto: A brief History of RL7m

Bellman Equation Derivation6m

Why Bellman Equations?5m

Optimal Policies7m

Optimal Value Functions5m

Using Optimal Value Functions to Get Optimal Policies8m

Week 3 Summary4m

3 readings

Module 3 Learning Objectives10m

Weekly Reading30m

Chapter Summary13m

2 practice exercises

[Practice] Value Functions and Bellman Equations45m

[Graded] Value Functions and Bellman Equations45m

Week

Week 4

4 hours to complete

Dynamic Programming

This week, you will learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming for industrial applications and problems. Further, you will learn about Generalized Policy Iteration as a common template for constructing algorithms that maximize reward. For this week’s graded assessment, you will implement an efficient dynamic programming agent in a simulated industrial control problem.

4 hours to complete

10 videos (Total 72 min), 3 readings, 2 quizzes

10 videos

Policy Evaluation vs. Control4m

Iterative Policy Evaluation8m

Policy Improvement4m

Policy Iteration8m

Flexibility of the Policy Iteration Framework4m

Efficiency of Dynamic Programming5m

Warren Powell: Approximate Dynamic Programming for Fleet Management (Short)7m

Warren Powell: Approximate Dynamic Programming for Fleet Management (Long)21m

Week 4 Summary2m

Congratulations!3m

3 readings

Module 4 Learning Objectives10m

Weekly Reading30m

Chapter Summary30m

1 practice exercise

Dynamic Programming45m

Reviews

4.8

604 reviews

5 stars
81.94%
4 stars
14.70%
3 stars
2.35%
2 stars
0.31%
1 star
0.67%

TOP REVIEWS FROM FUNDAMENTALS OF REINFORCEMENT LEARNING

by PVNov 9, 2019

I understood all the necessary concepts of RL. I've been working on RL for some time now, but thanks to this course, now I have more basic knowledge about RL and can't wait to watch other courses

by HSSep 19, 2019

One of the best courses I finished on Coursera, I really like the structure of the course. Textbook is also provided which really helps. Looking forward to next course in the series.

by RDOct 15, 2019

An excellent introduction to the subject of Reinforcement Learning, accompanied by a very clear text book. The python assignments in Jupyter notebooks are both informative and helpful.

by UJan 2, 2021

The book is essential reading. It took me longer than the estimates to do the reading and the programming assignments. I would have liked more gridworld examples to get a faster hang of it.

View all reviews

About the Reinforcement Learning Specialization

The Reinforcement Learning Specialization consists of 4 courses exploring the power of adaptive learning systems and artificial intelligence (AI).

Harnessing the full potential of artificial intelligence requires adaptive learning systems. Learn how Reinforcement Learning (RL) solutions help solve real-world problems through trial-and-error interaction by implementing a complete RL solution from beginning to end. By the end of this Specialization, learners will understand the foundations of much of modern probabilistic artificial intelligence (AI) and be prepared to take more advanced courses or to apply AI tools and ideas to real-world problems. This content will focus on “small-scale” problems in order to understand the foundations of Reinforcement Learning, as taught by world-renowned experts at the University of Alberta, Faculty of Science. The tools learned in this Specialization can be applied to game development (AI), customer interaction (how a website interacts with customers), smart assistants, recommender systems, supply chain, industrial control, finance, oil & gas pipelines, industrial control systems, and more.

Frequently Asked Questions

When will I have access to the lectures and assignments?
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
Is financial aid available?
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.

More questions? Visit the Learner Help Center.

Fundamentals of Reinforcement Learning

About this Course

What you will learn

Skills you will gain