This course covers the basics of financial impact estimation for machine learning models deployed in business processes. We will discuss the general approaches to financial estimation, consider the applications to credit scoring and marketing response models, and focus on the relationship between statistical model quality metrics and financial results, as well as the concepts of A/B testing and potential biases as they apply to historical data.
Offered By
Estimating ML-Models Financial Impact
HSE UniversityAbout this Course
3,245 recent views
Shareable Certificate
Earn a Certificate upon completion
100% online
Start instantly and learn at your own schedule.
Flexible deadlines
Reset deadlines in accordance to your schedule.
Intermediate Level
Basic experience with binary classification models, statistics and Python
Approx. 20 hours to complete
English
Subtitles: English
What you will learn
Estimating ML models financial impact for binary classification models
Design and evaluation of A/B tests
Reject inference
Skills you will gain
Model financial impactMetalearningA/B TestingBenefit curve analysis
Shareable Certificate
Earn a Certificate upon completion
100% online
Start instantly and learn at your own schedule.
Flexible deadlines
Reset deadlines in accordance to your schedule.
Intermediate Level
Basic experience with binary classification models, statistics and Python
Approx. 20 hours to complete
English
Subtitles: English
Offered by
HSE University
HSE University is one of the top research universities in Russia. Established in 1992 to promote new research and teaching in economics and related disciplines, it now offers programs at all levels of university education across an extraordinary range of fields of study including business, sociology, cultural studies, philosophy, political science, international relations, law, Asian studies, media and communicamathematics, engineering, and more.
Syllabus - What you will learn from this course
2 hours to complete
Project valuation: valuation metrics, planning and rules
Model development and deployment is always a project. During this week, we will discuss project valuation. We will consider the general financial metrics such as Net Present Value, Internal Rate of Return, and others.
2 hours to complete
5 videos (Total 40 min), 2 readings, 1 quiz
6 hours to complete
Model quality and decision making. Benefit curve
During the second week, we will focus on decision-making based on model predictions and the relationship between model quality and financial benefit. We will discuss how to plot benefit curves using different threshold decisions and optimize the financial results using threshold tuning.
6 hours to complete
7 videos (Total 57 min), 1 reading, 3 quizzes
4 hours to complete
Estimating model risk discounts
When a model is being used in the production environment for a long time, its quality can deteriorate. During this week, we will learn to calculate confidence intervals for model quality estimates and estimate potential negative financial effects.
4 hours to complete
6 videos (Total 26 min), 1 reading, 2 quizzes
4 hours to complete
A/B testing and financial result verification
A/B testing is a great way to verify our expectations concerning the financial effects. During this week, we will discuss the principles of A/B testing, its design, as well as the ways to assess its outcomes.
4 hours to complete
6 videos (Total 35 min), 1 reading, 2 quizzes
Frequently Asked Questions
When will I have access to the lectures and assignments?
What will I get if I purchase the Certificate?
Is financial aid available?
Will I earn university credit for completing the Course?
More questions? Visit the Learner Help Center.
