Want to make sense of the volumes of data you have collected? Need to incorporate data-driven decisions into your process? This course provides an overview of machine learning techniques to explore, analyze, and leverage data. You will be introduced to tools and algorithms you can use to create machine learning models that learn from data, and to scale those models up to big data problems. At the end of the course, you will be able to: • Design an approach to leverage data using the steps in the machine learning process. • Apply machine learning techniques to explore and prepare data for modeling. • Identify the type of machine learning problem in order to apply the appropriate set of techniques. • Construct models that learn from data using widely available open source tools. • Analyze big data problems using scalable machine learning algorithms on Spark. Software Requirements: Cloudera VM, KNIME, Spark
Confusion Matrix
Loading...
From the course by University of California San Diego
Machine Learning With Big Data
847 ratings
From the lesson
Evaluation of Machine Learning Models
Meet the Instructors
Mai NguyenLead for Data Analytics
San Diego Supercomputer Center
Ilkay AltintasChief Data Science Officer
San Diego Supercomputer Center
In addition to the evaluation matrix covered n the last lecture.
The performance of a classification model can also be evaluated
using a Confusion Matrix.
We will introduce the Confusion Matrix, in this lecture.
After this video you will be able to,
describe how a confusion matrix can be used to evaluate a classifier.
Interpret the confusion matrix of a model.
And relate accuracy to values in a confusion matrix.
Let's use our example again of predicting whether a given animal is a mammal or not.
Recall that this is a binary classification task, with the Class Label
being either Yes, indicating mammal or No indicating non-mammal.
Now let's review the different Types of Errors that you can get with
Classification.
If the True Label is Yes and
the Predicted Label is Yes, then this is a True Positive, abbreviated as TP.
This is a case where the label is correctly predicted as positive.
If the True Label is No and
the Predicted Label is No, then this is a True Negative, abbreviated as TN.
This is the case where the label is correctly predicted as negative.
If the True Label is No and
the Predicted Label is Yes, then this is a False Positive, abbreviated as FP.
This is the case where the label is incorrectly predicted as positive
when it should be negative.
If the True Label is Yes and the Predicted Label is No,
then this is a False Negative abbreviated as FN.
This is the case where the label is incorrectly predicted as negative,
when it should be positive.
A Confusion Matrix can be used to summarize the different types of
classification errors.
The True Positive cell corresponds to the samples that are correctly predicted as
positive by the model.
The True Negative cell corresponds to the samples that are correctly predicted
as negative.
The False Positive cell corresponds to samples that are incorrectly predicted
as positive.
The False Negative cell corresponds to samples that are incorrectly predicted
as negative.
Each cell has the count, or percentage of samples, with each type of errors.
Let's look at an example to see how a Confusion Matrix is filled in.
The table on the lists True Labels along with the models prediction for
a data set of ten samples.
We'll summarize this results in a Confusion Matrix.
First let's figure out the number of true positives.
We call that a true positive occurrence when the output is correctly predicted
as positive.
In other words, the true label is Yes and the model's prediction is also Yes.
In this example, there are three true positives as indicated by the red arrows.
We enter three and the true positive cell in the Confusion Matrix.
Now let's look at the true negatives.
A true negative occurs when the output is correctly predicted as negative.
In other words, the true label is No and the models prediction is also No.
In this example,
there are four true negatives as indicated by the green arrows.
We enter four in the true negative cell in the Confusion Matrix.
What about false negatives?
A false negative occurs when the output is incorrectly predicted as negative,
when it should be positive.
That is the true label is Yes, and the model's prediction is No.
In this example,
there are two false negatives as indicated by the purple arrows.
We enter two in the false negative cell in the Confusion Matrix.
Finally we need to look at false positives.
A false positive occurs,
when the input is incorrectly predicted as positive, when it should be negative.
That is the true label is No and models prediction is Yes.
In this example there is one false positive as indicated by the yellow arrow.
We enter one in the false positive cell in the Confusion Matrix.
This is our complete Confusion Matrix for this example.
We see that the sum of the numbers in the cells add up to ten,
which is the number of samples in our dataset.
Note that the diagonal values, true positives and
true negatives are samples with Correct Predictions.
In our example, these values sum up to 7.
Meaning that 7 out of 10 samples were correctly predicted by the model.
The higher the sum of the diagonal values, the better the performance of the model.
The off diagonal values capture the misclassified samples.
Where the model's predictions do not match the true labels.
In this example these values indicate that there were three misclassifications.
Smaller values for
the off diagonal cells in the confusion matrix indicate better model performance.
Note that the diagonal values, true positives and
true negatives are samples with correct predictions.
In our example, these values sum up to 7 meaning
that 7 out of 10 samples were correctly predicted by the model.
The higher the sum of the diagonal values, the better the performance of the model.
You may have noticed that the diagonal values are related to Accuracy Rate.
Recall that accuracy is defined as the sum of two positives and
true negatives, divided by all samples.
The sum of two positives and
two negatives is the sum of the diagonal values in a confusion matrix.
The sum of the diagonal values in a confusion matrix,
divided by the total number of samples gives you the Accuracy Rate.
Similarly, the off diagonal values are related to the Error Rate.
Recall that the Error Rate is the opposite of the Accuracy Rate.
The sum of the off diagonal values in a confusion matrix,
divided by the total number of samples, gives you the Error Rate.
Looking at the values in the Confusion Matrix can help you understand
the kind of Misclassifications that your model is making.
A high value for this cell indicated by the yellow arrow,
means that classifying the Positive class is problematic for the model.
A high value for the cell indicated by the orange arrow on the other hand,
means that classifying the Negative class is problematic for the model.
In summary, the Confusion Matrix is a table
used to summarize the different types of errors for a classifier.
The values in a Confusion Matrix can be used to evaluate the performance of
a classifier and are related to evaluation metrics, such as accuracy and error rates.
They also indicate what other types of misclassifications the model is making.
Note that in some implementations of a Confusion Matrix, the true and
predictive labels are switched.
Be sure to review the documentation for the software you're using to generate
a Confusion Matrix to understand what each cell specifies.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
