When we start estimating, we could simply say, okay here's the explanatory variables and here is a response variable, let's run a regression or could we be more cautious and we can start at the very simple model. As we go through this course, we will see one of the things we have to be careful about is use fewer variables because otherwise we run into a danger of over-fitting and explaining things which may not be there. So lets start with a very simple model, which is let's say, the median value is equal to something, let's call it b0 plus b1 times crime. So we are saying that the price, the median price in a particular neighborhood is a function of the crime in neighborhood. But it may not be perfect. So we act clever and add an error out there. Okay. So, this is the model we want to estimate. All right. You might say okay, it could be crime squared. You could say it could be the log of the crime. You could substitute anything you want for crime and the model is still, we call a linear model. We also say that for each observation in your data, we can rewrite it. Let's say the first median value, we'll call it medv1 is equal to b0 plus b1 times the crime in that area plus the error we make for that data point. So the big difference between the first equation and the second equation, the first is a general equation, the second one is applying the equation to the first data point and this way we could apply it to the second point, the third point, the fourth point or the eighth point. For the ith point, it will be nothing but medvi, the median value of the ith area is equal to b0 plus b1 times the crime in the ith area plus the error for the ith area. Okay. Of course, we realize that we cannot achieve a perfect fit, right? Are we going to achieve a 100% fit? Probably not, depends on what kind of fit we want to achieve. One way of looking at it is just writing the error, it's the errori is equal to, you take everything to the left and so this is the actual value minus the predicted value. Okay. So what I have written is the error you are making is that difference between the actual response minus the predicted response. So our objective is to minimize this error in some sense. Is that only one way to do it? No. So we're going to use one method which is called the least square method which is popular because it has got some very nice properties. But there are other methods right and we won't talk about them much in this course but like maximum likelihood method, ridge regression and we will see a methodology which uses principal components, when we come to principal components later on. So what we're really trying to do is look at the difference between the actual minus predicted value and minimize the sum of the square of these errors. So your objective is to minimize what we call the least squares. So find the fit which gives you the least square. So you ask me what is a fit right? A fit is from the data, I want to estimate values of b0 and b1. Okay. So that's the problem we are trying to solve. Find me the values of b0 and b1 such that the square of the error, the sum of the squares of the errors is minimized okay, and that's called ordinary linear regression. By the way, this is called a simple regression model and let me assure you there's nothing simple about it. It's called simple because there is only one explanatory variable. So here's a model visualization. We're trying to explain the median value of houses as a function of the crime rate and the crime rate is on the x-axis, the median value is on the y-axis. This graph, what I've done is I've just, you remember the correlation graphs, I just snipped it and enlarged it. From this graph, you might guess that this line has an intercept around 25,000 and it has a slope and this slope of the line actually goes for an increase of 50 units in the crime rate. The median value drops almost to zero. So we expect the slope is about minus 0.5. For 50 units of change in the crime rate, the median value goes from 25 to 0. Close to zero it's not exactly a zero. So what do we really expect is a line which has an intercept of 25. This is the intercept and the slope of the line as minus half, as the crime rate increases, we expect the values to drop. Okay. Now let's go to rattle and see how to estimate. Okay. So this is the first time we're doing it. So I might as well go to the rattle and do it. Okay. Let's go to rattle. Let's go to data and at this point, I want to ignore all the explanatory variables and I just want to retain the median value. If you notice, I have selected it as the target variable and I want to make the explanatory variable crime. Okay, and execute. Never forget to execute. Then, you go to model. Various models are there. We will learn about some of them but what we want is a linear model because the model is a line. You'd select linear. Interestingly, what it does is, it says it's numeric because it recognizes that the response variable which is the price is numeric. If it is not numeric, it'll probably choose one of these other methods and you say execute. So when you execute, it estimates the line. It has not yet produced any plots. All it has done is execute the line. I come back and explain what this line is. Okay. Let's go back to the PowerPoint. So what it does is given us an output and we will go through this output and also I'll do some plot. So we'll do three things. We will try to understand very briefly how to interpret this output. We will then visually examine how we are doing a great job and then, there is a way of looking at the residuals I won't spend too long about it, but I'll show you how to look at it. So there are three things I'm going to do. Is this a good fit? What is the fit? How do you examine it? How do you get residuals? Okay.
