Learn how to analyze data using Python. This course will take you from the basics of Python to exploring many different types of data. You will learn how to prepare data for analysis, perform simple statistical analyses, create meaningful data visualizations, predict future trends from data, and more! Topics covered: 1) Importing Data sets 2) Cleaning the Data 3) Data frame manipulation 4) Summarizing the Data 5) Building machine learning Regression models 6) Building data pipelines Data Analysis with Python will be delivered through lecture, lab, and assignments. It includes following parts: Data Analysis libraries: will learn to use Pandas, Numpy and Scipy libraries to work with a sample dataset. We will introduce you to pandas, an open-source library, and we will use it to load, manipulate, analyze, and visualize cool datasets. Then we will introduce you to another open-source library, scikit-learn, and we will use some of its machine learning algorithms to build smart models and make cool predictions. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge. To find out more about IBM digital badges follow the link ibm.biz/badging.
Overfitting, Underfitting and Model Selection
Loading...
From the course by IBM
Data Analysis with Python
87 ratings
From the lesson
Model Evaluation
Meet the Instructors
Joseph SantarcangeloPh.D., Data Scientist at IBM
If you recall, in the last module,
we discussed polynomial regression.
In this section, we will discuss how to pick
the best polynomial order and problems
that arise when selecting the wrong order polynomial.
Consider the following function,
we assume the training points come from a polynomial function plus some noise.
The goal of Model Selection is to determine the order of
the polynomial to provide the best estimate of the function y(x).
If we try and fit the function with a linear function,
the line is not complex enough to fit the data.
As a result, there are many errors.
This is called underfitting,
where the model is too simple to fit the data.
If we increase the order of the polynomial,
the model fits better, but the model is
still not flexible enough and exhibits underfitting.
This is an example of the 8th order polynomial used to fit the data.
We see the model does well at fitting the data and estimating
the function even at the inflection points.
Increasing it to a 16th order polynomial,
the model does extremely well at tracking
the training point but performs poorly at estimating the function.
This is especially apparent where there is little training data.
The estimated function oscillates not tracking the function.
This is called overfitting,
where the model is too flexible and fits the noise rather than the function.
Let's look at a plot of the mean square error for
the training and testing set of different order polynomials.
The horizontal axis represents the order of the polynomial.
The vertical axis is the mean square error.
The training error decreases with the order of the polynomial.
The test error is a better means of estimating the error of a polynomial.
The error decreases 'til the best order of the polynomial
is determined then the error begins to increase.
We select the order that minimizes the test error.
In this case, it was eight.
Anything on the left would be considered underfitting,
anything on the right is overfitting.
If we select the best order of the polynomial,
we will still have some errors.
If you recall the original expression for the training points we see a noise term.
This term is one reason for the error.
This is because the noise is random and we can't predict it.
This is sometimes referred to as an irreducible error.
There are other sources of errors as well.
For example, our polynomial assumption may be wrong.
Our sample points may have come from a different function.
For example, in this plot,
the data is generated from a sine wave.
The polynomial function does not do a good job at fitting the sine wave.
For real data, the model may be too difficult to
fit or we may not have the correct type of data to estimate the function.
Let's try different order polynomials on the real data using horsepower.
The red points represent the training data.
The green points represent the test data.
If we just use the mean of the data,
our model does not perform well.
A linear function does fit the data better.
A second order model looks similar to the linear function.
A third order function also appears to increase,
like the previous two orders.
Here, we see a fourth order polynomial.
At around 200 horsepower,
the predicted price suddenly decreases.
This seems erroneous.
Let's use R-squared to see if our assumption is correct.
The following is a plot of the R-squared value.
The horizontal axis represents the order polynomial models.
The closer the R-squared is to one the more accurate the model is.
Here, we see the R-squared is optimal when the order of the polynomial is three.
The R-squared drastically decreases when the order is increased to four,
validating our initial assumption.
We can calculate different R-squared values as follows.
First, we create an empty list to store the values.
We create a list containing different polynomial orders.
We then iterate through the list using a loop.
We create a polynomial feature object with the order of the polynomial as a parameter.
We transform the training and test data into a polynomial using the fit transform method.
We fit the regression model using the transform data.
We then calculate the R-squared using the test data and store it in the array.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
