Illinois Tech

Statistical Learning


Gain insight into a topic and learn the fundamentals

Shahrzad Jamshidi

Instructor: Shahrzad Jamshidi

Intermediate level

Recommended experience

115 hours to complete
3 weeks at 38 hours a week
Flexible schedule
Learn at your own pace
Progress towards a degree

See how employees at top companies are mastering in-demand skills


Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV

Share it on social media and in your performance review


There are 9 modules in this course

Welcome to Statistical Learning! In this course, we will cover the topics: Statistical Learning: Terminology and Ideas, Linear Regression Methods, Linear Classification Methods, Basis Expansion Methods, Kernel Smoothing Methods, Model Assessment and Selection, Maximum Likelihood Inference, and Advanced Topics. Module 1 offers an in-depth exploration of statistical learning, beginning with the rationale behind choosing a pre-defined family of functions and optimizing the expected prediction error (EPE). It covers the essentials of statistical learning, including the loss function, the bias-variance tradeoff in model selection, and the significance of model evaluation. This module also distinguishes between supervised and unsupervised learning, discusses various types of statistical learning models and data representation, and delves into the three core elements of a statistical learning problem, providing a comprehensive introduction to this field.

What's included

8 videos5 readings4 assignments1 discussion prompt1 ungraded lab

Welcome to Module 2 of Math 569: Statistical Learning. Here, we explore what is arguably the foundational model of the field: linear regression. This simple yet highly useful model helps us better understand the statistical learning problem discussed in Module 1. In Lesson 1, we'll carefully review what linear regression aims to do, how we construct the model's parameters with a given dataset, and what kinds of statistical tests we can perform on our estimated coefficients. In Lesson 2, we’ll cover a method known as Subset Selection, which aims to improve linear regression by eliminating unimpactful independent variables. In Lesson 3, we explore introducing bias into the linear regression model with two regularization methods: Ridge Regression and LASSO. These methods utilize a hyperparameter, a key concept in this course, to limit the growth of the coefficients. This is the source of the bias and will help us understand why a biased estimator can outperform our unbiased estimator for the coefficients of linear regression in Lesson 1. Finally, Lesson 4 introduces the concept of data transformations, which allow one to address complexities within a dataset. It also provides a simple way of converting a linear model to a nonlinear model.

What's included

10 videos6 readings5 assignments6 ungraded labs

Welcome to Module 3 of Math 569: Statistical Learning, where we delve into linear classification. In Lesson 1, we explore how linear regression, typically used for predicting continuous outcomes, can be adapted for classification tasks-predicting discrete categories. We'll cover the conversion of categorical data into a numerical format suitable for classification and introduce essential classification metrics such as accuracy, precision, and recall. In Lesson 2, we'll explore Linear Discriminant Analysis (LDA) as an alternative method for constructing linear classifications. This method introduces the notion that classification maximizes the probability of a category given a data point, a framing we will revisit later in the course. Maximizing the likelihood of classification, given some simplifying assumptions, leads to a linear model that can also reduce the dimensionality of the problem. Finally, in Lesson 3, we will cover logistic regression, which is constructed by assuming the log-likelihood odds are linear models. The outcome, similar to LDA, produces a linear decision boundary.

What's included

5 videos5 readings4 assignments6 ungraded labs

Welcome to Module 4 of Math 569: Statistical Learning, focusing on advanced methods in statistical modeling. This module starts with an introduction to Basis Expansion Methods, exploring how these techniques enhance linear models by incorporating non-linear relationships. We then delve into Piecewise Polynomials, discussing their utility in capturing varying trends across different segments of data. In Lesson 2, we explore Smoothing Splines, emphasizing their role in effectively balancing model fit and complexity. Lastly, Lesson 3 covers Regularization and Kernel Functions, elaborating on how these concepts contribute to constructing more complex models without significantly increasing computational complexity.

What's included

5 videos5 readings4 assignments6 ungraded labs

Welcome to Module 5 of Math 569: Statistical Learning, dedicated to advanced techniques in non-linear data modeling. In Lesson 1, we delve into Kernel Smoothers, exploring how they make predictions based on local data and their comparison to k-Nearest Neighbors (kNN) models. Lesson 2 focuses on Local Regression, particularly Local Linear Regression (LLR) and Local Polynomial Regression (LPR). We'll examine how LLR overcomes some kernel smoothing limitations and how LPR provides flexibility in capturing local data structure. The module emphasizes the adaptiveness of these techniques for complex data relationships and addresses the challenges in selecting hyperparameters and computational demands, especially for large datasets.

What's included

3 videos4 readings3 assignments4 ungraded labs

Module 6 of Math 569: Statistical Learning delves into model evaluation and model selection via hyperparameter choice. It begins with an understanding of Bias-Variance Decomposition, highlighting the trade-off between model simplicity and accuracy. The module then explores model complexity, offering strategies for balancing this complexity with predictive performance. Building on the importance of balancing model complexity with performance, we move on to cover model selection metrics, namely: AIC, BIC, and MDL. These are information-theoretic metrics that balance error with model complexity, such as the number of parameters. Finally, the module concludes with lessons on estimating test error without a testing set, using concepts like VC Dimension, Cross-Validation, and Bootstrapping. This module is pivotal for mastering model evaluation and selection in statistical learning.

What's included

8 videos7 readings7 assignments9 ungraded labs

Module 7 of Math 569: Statistical Learning introduces advanced inferential techniques. Lesson 1 focuses on Maximum Likelihood Inference, explaining how to find optimal model parameters by maximizing the likelihood function. This method is pivotal in estimating parameters for which a dataset is most likely. Lesson 2 dives into Bayesian Inference, contrasting it with frequentist approaches. It covers Bayes' Theorem, which integrates prior beliefs with new evidence to update beliefs dynamically. The module thoroughly discusses the process of Bayesian modeling, including the construction and updating of models using prior and posterior distributions. This module is crucial for understanding complex inference methods in statistical learning.

What's included

4 videos4 readings4 assignments2 ungraded labs

Module 8 of Math 569: Statistical Learning covers diverse advanced machine learning techniques. It begins with Decision Trees, focusing on their structure and application in both classification and regression tasks. Next, it explores Support Vector Machines (SVM), detailing their function in creating optimal decision boundaries. The module then examines k-Means Clustering, an unsupervised learning method for data grouping. Finally, it concludes with Neural Networks, discussing their architecture and role in complex pattern recognition. Each lesson offers a deep dive into these techniques, showcasing their unique advantages and applications in statistical learning.

What's included

6 videos5 readings5 assignments8 ungraded labs

This module contains the summative course assessment that has been designed to evaluate your understanding of the course material and assess your ability to apply the knowledge you have acquired throughout the course. Be sure to review the course material thoroughly before taking the assessment.

What's included

1 assignment


Shahrzad Jamshidi
Illinois Tech
1 Course529 learners

Offered by

Illinois Tech

Recommended if you're interested in Probability and Statistics

Get a head start on your degree

This course is part of the following degree programs offered by Illinois Tech. If you are admitted and enroll, your coursework can count toward your degree learning and your progress can transfer with you.

Why people choose Coursera for their career

Felipe M.
Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."
Jennifer J.
Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."
Larry W.
Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."
Chaitanya A.
"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

New to Probability and Statistics? Start here.


Open new doors with Coursera Plus

Unlimited access to 7,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription

Advance your career with an online degree

Earn a degree from world-class universities - 100% online

Join over 3,400 global companies that choose Coursera for Business

Upskill your employees to excel in the digital economy

Frequently asked questions