Estimating ML-Models Financial Impact

This course covers the basics of financial impact estimation for machine learning models deployed in business processes. We will discuss the general approaches to financial estimation, consider the applications to credit scoring and marketing response models, and focus on the relationship between statistical model quality metrics and financial results, as well as the concepts of A/B testing and potential biases as they apply to historical data.

Multiple courses focus on building machine learning models and assessing their predictive power. However, much less attention is usually paid to explaining how the model quality translates into financial results. Even more so, decision strategies relying on model predictions are normally not covered in great detail. In this course, we will focus on the step when we already have a ML model and want to estimate the expected financial results, and verify the model by running either an A/B test or a backtest. In addition, we will learn how to tune threshold decision rules for model probabilities, thereby improving financial results, as well as account for model uncertainty or biases in historical data that may tamper with our financial estimates. We will analyze the binary classification case, which is the most common type of a ML task. After completing this course, you, as a data scientist, will be able to come up with better arguments when explaining the value of your machine learning models to your leadership. If your role in the company gravitates toward business processes, you will gain a better understanding of how machine learning models can have an impact on the financial results.