Reinforcement Learning is a subfield of Machine Learning, but is also a general purpose formalism for automated decision-making and AI. This course introduces you to statistical learning techniques where an agent explicitly takes actions and interacts with the world. Understanding the importance and challenges of learning agents that make decisions is of vital importance today, with more and more companies interested in interactive agents and intelligent decision-making.
This course is part of the Reinforcement Learning Specialization
Fundamentals of Reinforcement Learning
Offered By
University of Alberta
Alberta Machine Intelligence Institute
About this Course
180,115 recent views
What you will learn
Formalize problems as Markov Decision Processes
Understand basic exploration methods and the exploration / exploitation tradeoff
Understand value functions, as a general-purpose tool for optimal decision-making
Know how to implement dynamic programming as an efficient solution approach to an industrial control problem
Skills you will gain
Artificial Intelligence (AI)Machine LearningReinforcement LearningFunction ApproximationIntelligent Systems
100% online
Start instantly and learn at your own schedule.
Course 1 of 4 in the
Flexible deadlines
Reset deadlines in accordance to your schedule.
Intermediate Level
Probabilities & Expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), implementing algorithms from pseudocode.
Approx. 19 hours to complete
Suggested: 4-6 hours/week
English
Subtitles: English
Syllabus - What you will learn from this course
1 hour to complete
Welcome to the Course!
Welcome to: Fundamentals of Reinforcement Learning, the first course in a four-part specialization on Reinforcement Learning brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, get a flavour of what the course has in store for you, and be given an in-depth roadmap to help make your journey through this specialization as smooth as possible.
1 hour to complete
4 videos (Total 20 min), 2 readings
4 videos
Course Introduction5m
Meet your instructors!8m
Your Specialization Roadmap3m
2 readings
Reinforcement Learning Textbook10m
Read Me: Pre-requisites and Learning Objectives10m
7 hours to complete
The K-Armed Bandit Problem
For the first week of this course, you will learn how to understand the exploration-exploitation trade-off in sequential decision-making, implement incremental algorithms for estimating action-values, and compare the strengths and weaknesses to different algorithms for exploration. For this week’s graded assessment, you will implement and test an epsilon-greedy agent.
7 hours to complete
8 videos (Total 46 min), 3 readings, 2 quizzes
8 videos
Learning Action Values4m
Estimating Action Values Incrementally5m
What is the trade-off?7m
Optimistic Initial Values6m
Upper-Confidence Bound (UCB) Action Selection5m
Jonathan Langford: Contextual Bandits for Real World Reinforcement Learning8m
Week 1 Summary3m
3 readings
Module 2 Learning Objectives10m
Weekly Reading30m
Chapter Summary30m
1 practice exercise
Exploration/Exploitation45m
3 hours to complete
Markov Decision Processes
When you’re presented with a problem in industry, the first and most important step is to translate that problem into a Markov Decision Process (MDP). The quality of your solution depends heavily on how well you do this translation. This week, you will learn the definition of MDPs, you will understand goal-directed behavior and how this can be obtained from maximizing scalar rewards, and you will also understand the difference between episodic and continuing tasks. For this week’s graded assessment, you will create three example tasks of your own that fit into the MDP framework.
3 hours to complete
7 videos (Total 36 min), 2 readings, 2 quizzes
7 videos
Examples of MDPs4m
The Goal of Reinforcement Learning3m
Michael Littman: The Reward Hypothesis12m
Continuing Tasks5m
Examples of Episodic and Continuing Tasks3m
Week 2 Summary1m
2 readings
Module 3 Learning Objectives10m
Weekly Reading30m
1 practice exercise
MDPs45m
3 hours to complete
Value Functions & Bellman Equations
Once the problem is formulated as an MDP, finding the optimal policy is more efficient when using value functions. This week, you will learn the definition of policies and value functions, as well as Bellman equations, which is the key technology that all of our algorithms will use.
3 hours to complete
9 videos (Total 56 min), 3 readings, 2 quizzes
9 videos
Value Functions6m
Rich Sutton and Andy Barto: A brief History of RL7m
Bellman Equation Derivation6m
Why Bellman Equations?5m
Optimal Policies7m
Optimal Value Functions5m
Using Optimal Value Functions to Get Optimal Policies8m
Week 3 Summary4m
3 readings
Module 4 Learning Objectives10m
Weekly Reading30m
Chapter Summary13m
2 practice exercises
Value Functions and Bellman Equations45m
Value Functions and Bellman Equations45m
7 hours to complete
Dynamic Programming
This week, you will learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming for industrial applications and problems. Further, you will learn about Generalized Policy Iteration as a common template for constructing algorithms that maximize reward. For this week’s graded assessment, you will implement an efficient dynamic programming agent in a simulated industrial control problem.
7 hours to complete
10 videos (Total 72 min), 3 readings, 2 quizzes
10 videos
Iterative Policy Evaluation8m
Policy Improvement4m
Policy Iteration8m
Flexibility of the Policy Iteration Framework4m
Efficiency of Dynamic Programming5m
Warren Powell: Approximate Dynamic Programming for Fleet Management (Short)7m
Warren Powell: Approximate Dynamic Programming for Fleet Management (Long)21m
Week 4 Summary2m
Congratulations!3m
3 readings
Module 5 Learning Objectives10m
Weekly Reading30m
Chapter Summary30m
1 practice exercise
Dynamic Programming45m
Reviews
4.8
TOP REVIEWS FROM FUNDAMENTALS OF REINFORCEMENT LEARNING
by PVNov 9, 2019
I understood all the necessary concepts of RL. I've been working on RL for some time now, but thanks to this course, now I have more basic knowledge about RL and can't wait to watch other courses
by ABSep 6, 2019
Concepts are bit hard, but it is nice if you undersand it well, espically the bellman and dynamic programming.\n\nSometimes, visualizing the problem is hard, so need to thoroghly get prepared.
by RDOct 15, 2019
An excellent introduction to the subject of Reinforcement Learning, accompanied by a very clear text book. The python assignments in Jupyter notebooks are both informative and helpful.
by HSSep 19, 2019
One of the best courses I finished on Coursera, I really like the structure of the course. Textbook is also provided which really helps. Looking forward to next course in the series.
by HCFeb 18, 2020
Excellent and well done course on some of the basics of RL. Good mix of lectures, reading, quizzes and programming assignments. Also a good balance between pure theory and examples.
by NSJan 1, 2020
Very well designed, it is clear that a lot of thought was put into the course. Also, I really liked the clarity regarding the learning objectives and the emphasis on understanding.
by NSAug 3, 2019
The ideal course to go with the book Reinforcement Learning: An Introduction. The quizzes and coding workshops are pitched just right in my opinion, neither too easy nor too hard.
by SKDec 9, 2019
A great introduction to RL. Credit goes to the instructors Mr and Mrs white for keep in it as simple as possible. Understanding the math behind RL is the key for the RL adventure.
About University of Alberta
UAlberta is considered among the world’s leading public research- and teaching-intensive universities. As one of Canada’s top universities, we’re known for excellence across the humanities, sciences, creative arts, business, engineering and health sciences.
About Alberta Machine Intelligence Institute
The Alberta Machine Intelligence Institute (Amii) is home to some of the world’s top talent in machine intelligence. We’re an Alberta-based
research institute that pushes the bounds of academic knowledge and guides business understanding of artificial intelligence and machine learning.
About the Reinforcement Learning Specialization
The Reinforcement Learning Specialization consists of 4 courses exploring the power of adaptive learning systems and artificial intelligence (AI).
Harnessing the full potential of artificial intelligence requires adaptive learning systems. Learn how Reinforcement Learning (RL) solutions help solve real-world problems through trial-and-error interaction by implementing a complete RL solution from beginning to end.
By the end of this Specialization, learners will understand the foundations of much of modern probabilistic artificial intelligence (AI) and be prepared to take more advanced courses or to apply AI tools and ideas to real-world problems. This content will focus on “small-scale” problems in order to understand the foundations of Reinforcement Learning, as taught by world-renowned experts at the University of Alberta, Faculty of Science.
The tools learned in this Specialization can be applied to game development (AI), customer interaction (how a website interacts with customers), smart assistants, recommender systems, supply chain, industrial control, finance, oil & gas pipelines, industrial control systems, and more.
Frequently Asked Questions
When will I have access to the lectures and assignments?
Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
What is the refund policy?
Is financial aid available?
More questions? Visit the Learner Help Center.