This course covers the basics of financial impact estimation for machine learning models deployed in business processes. We will discuss the general approaches to financial estimation, consider the applications to credit scoring and marketing response models, and focus on the relationship between statistical model quality metrics and financial results, as well as the concepts of A/B testing and potential biases as they apply to historical data.
Offered By
Estimating ML-Models Financial Impact
National Research University Higher School of EconomicsAbout this Course
Basic experience with binary classification models, statistics and Python
What you will learn
Estimating ML models financial impact for binary classification models
Design and evaluation of A/B tests
Reject inference
Skills you will gain
Basic experience with binary classification models, statistics and Python
Offered by

National Research University Higher School of Economics
National Research University - Higher School of Economics (HSE) is one of the top research universities in Russia. Established in 1992 to promote new research and teaching in economics and related disciplines, it now offers programs at all levels of university education across an extraordinary range of fields of study including business, sociology, cultural studies, philosophy, political science, international relations, law, Asian studies, media and communicamathematics, engineering, and more.
Syllabus - What you will learn from this course
Project valuation: valuation metrics, planning and rules
Model development and deployment is always a project. During this week, we will discuss project valuation. We will consider the general financial metrics such as Net Present Value, Internal Rate of Return, and others.
Model quality and decision making. Benefit curve
During the second week, we will focus on decision-making based on model predictions and the relationship between model quality and financial benefit. We will discuss how to plot benefit curves using different threshold decisions and optimize the financial results using threshold tuning.
Estimating model risk discounts
When a model is being used in the production environment for a long time, its quality can deteriorate. During this week, we will learn to calculate confidence intervals for model quality estimates and estimate potential negative financial effects.
A/B testing and financial result verification
A/B testing is a great way to verify our expectations concerning the financial effects. During this week, we will discuss the principles of A/B testing, its design, as well as the ways to assess its outcomes.
Frequently Asked Questions
When will I have access to the lectures and assignments?
What will I get if I purchase the Certificate?
Is financial aid available?
More questions? Visit the Learner Help Center.