In addition to the evaluation matrix covered n the last lecture. The performance of a classification model can also be evaluated using a Confusion Matrix. We will introduce the Confusion Matrix, in this lecture. After this video you will be able to, describe how a confusion matrix can be used to evaluate a classifier. Interpret the confusion matrix of a model. And relate accuracy to values in a confusion matrix. Let's use our example again of predicting whether a given animal is a mammal or not. Recall that this is a binary classification task, with the Class Label being either Yes, indicating mammal or No indicating non-mammal. Now let's review the different Types of Errors that you can get with Classification. If the True Label is Yes and the Predicted Label is Yes, then this is a True Positive, abbreviated as TP. This is a case where the label is correctly predicted as positive. If the True Label is No and the Predicted Label is No, then this is a True Negative, abbreviated as TN. This is the case where the label is correctly predicted as negative. If the True Label is No and the Predicted Label is Yes, then this is a False Positive, abbreviated as FP. This is the case where the label is incorrectly predicted as positive when it should be negative. If the True Label is Yes and the Predicted Label is No, then this is a False Negative abbreviated as FN. This is the case where the label is incorrectly predicted as negative, when it should be positive. A Confusion Matrix can be used to summarize the different types of classification errors. The True Positive cell corresponds to the samples that are correctly predicted as positive by the model. The True Negative cell corresponds to the samples that are correctly predicted as negative. The False Positive cell corresponds to samples that are incorrectly predicted as positive. The False Negative cell corresponds to samples that are incorrectly predicted as negative. Each cell has the count, or percentage of samples, with each type of errors. Let's look at an example to see how a Confusion Matrix is filled in. The table on the lists True Labels along with the models prediction for a data set of ten samples. We'll summarize this results in a Confusion Matrix. First let's figure out the number of true positives. We call that a true positive occurrence when the output is correctly predicted as positive. In other words, the true label is Yes and the model's prediction is also Yes. In this example, there are three true positives as indicated by the red arrows. We enter three and the true positive cell in the Confusion Matrix. Now let's look at the true negatives. A true negative occurs when the output is correctly predicted as negative. In other words, the true label is No and the models prediction is also No. In this example, there are four true negatives as indicated by the green arrows. We enter four in the true negative cell in the Confusion Matrix. What about false negatives? A false negative occurs when the output is incorrectly predicted as negative, when it should be positive. That is the true label is Yes, and the model's prediction is No. In this example, there are two false negatives as indicated by the purple arrows. We enter two in the false negative cell in the Confusion Matrix. Finally we need to look at false positives. A false positive occurs, when the input is incorrectly predicted as positive, when it should be negative. That is the true label is No and models prediction is Yes. In this example there is one false positive as indicated by the yellow arrow. We enter one in the false positive cell in the Confusion Matrix. This is our complete Confusion Matrix for this example. We see that the sum of the numbers in the cells add up to ten, which is the number of samples in our dataset. Note that the diagonal values, true positives and true negatives are samples with Correct Predictions. In our example, these values sum up to 7. Meaning that 7 out of 10 samples were correctly predicted by the model. The higher the sum of the diagonal values, the better the performance of the model. The off diagonal values capture the misclassified samples. Where the model's predictions do not match the true labels. In this example these values indicate that there were three misclassifications. Smaller values for the off diagonal cells in the confusion matrix indicate better model performance. Note that the diagonal values, true positives and true negatives are samples with correct predictions. In our example, these values sum up to 7 meaning that 7 out of 10 samples were correctly predicted by the model. The higher the sum of the diagonal values, the better the performance of the model. You may have noticed that the diagonal values are related to Accuracy Rate. Recall that accuracy is defined as the sum of two positives and true negatives, divided by all samples. The sum of two positives and two negatives is the sum of the diagonal values in a confusion matrix. The sum of the diagonal values in a confusion matrix, divided by the total number of samples gives you the Accuracy Rate. Similarly, the off diagonal values are related to the Error Rate. Recall that the Error Rate is the opposite of the Accuracy Rate. The sum of the off diagonal values in a confusion matrix, divided by the total number of samples, gives you the Error Rate. Looking at the values in the Confusion Matrix can help you understand the kind of Misclassifications that your model is making. A high value for this cell indicated by the yellow arrow, means that classifying the Positive class is problematic for the model. A high value for the cell indicated by the orange arrow on the other hand, means that classifying the Negative class is problematic for the model. In summary, the Confusion Matrix is a table used to summarize the different types of errors for a classifier. The values in a Confusion Matrix can be used to evaluate the performance of a classifier and are related to evaluation metrics, such as accuracy and error rates. They also indicate what other types of misclassifications the model is making. Note that in some implementations of a Confusion Matrix, the true and predictive labels are switched. Be sure to review the documentation for the software you're using to generate a Confusion Matrix to understand what each cell specifies.
