We will now discuss the general autoregressive process of order p. As in the AR1, we're going to think about expressing current values of the time series in terms of past values of the time series. In this case, we're going to think about expressing the current time in terms of the past p-values of the time series process. That's why it's called an autoregressive model of order p. The order tells you how many lags you are going to be considering. We're going to then assume a process that is going to have this structure. Again, as I said, we are regressing on the past p-values. As before, in this case, we're going to assume that the epsilon t's are independent identically distributed random variables with a normal distribution centered at zero and variance v. This is the assumption that we are going to use here. As you see now, the number of parameters has increased. We had one coefficient before, now we're going to have p coefficients, and we're going to have also the variance of the process. One thing that is very important used to characterize and understand the properties of autoregressive processes is the so-called characteristic polynomial. The AR characteristic polynomial is a polynomial, I'm going to denote it like this, and it's going to be a function of the phi coefficients here. It's going to look like a polynomial here, and is a polynomial of order p. Here, U is any complex valued number. Why do we study this polynomial? This polynomial tells us a lot about the process and a lot about the properties of this process. One of the main characteristics is it allows us to think about things like quasi-periodic behavior, whether it's present or not in a particular AR p process. It allows us to think about whether a process is stationary or not, depending on some properties related to this polynomial. In particular, we are going to say that the process is stable here. This is the stability condition. If all the roots of this polynomial have a modulus, that is that they all have modulus that are greater than one, if, I'm going to write it like this, Phi of U, this polynomial is zero for a root, so for any value of U such that this happens, then we say that the process is stable. For any of the roots, it has to be the case that the modulus of that root, they have to be all outside the unit circle. When a process is stable, it's also going to be stationary. In this case, if the process is stable, then we have a stationary process. This is going to characterize also the stationarity of the process in terms of the roots of the characteristic polynomial. Once the process is stationary, and if all the roots of the characteristic polynomial are outside the unit circle, then we will be able to write this process in terms of an infinite order moving average process. In this case, if the process is stable, then we are going to be able to write it like this. I'm sorry, this should be epsilon t. I am going to have an infinite order polynomial here on B, the backshift operator that I can write down just as the sum, j goes from zero to infinity. Here Psi_0 is one. Then there is another condition on the Psi's for this to happen. We have to have finite sum of these on these coefficients. Once again, if the process is stable, then it would be stationary and we will be able to write down the AR as an infinite order moving average process here. If you recall, B is the backshift operator. Again, if I apply this to y_t, I'm just going to get y_t minus j. I can write down Psi of B, as 1 plus Psi_1 B, B squared, and so on. It's an infinite order process. The AR characteristic polynomial can also be written in terms of the reciprocal roots of the polynomial. So instead of considering the roots, we can consider the reciprocal roots. In that case, let's say the Phi of u for Alpha 1, Alpha 2, and so on. The reciprocal roots. Why do we care about all these roots? Why do we care about this structure? Again, we will be able to understand some properties of the process based on these roots as we will see. We will now discuss another important representation of the AR(P) process, one that is based on a state-space representation of the process. Again, we care about this type of representations because they allow us to study some important properties of the process. In this case, our state-space or dynamic linear model representation, we will make some connections with these representations later when we talk about dynamic linear models, is given as follows for an AR(P). I have my y_t. I can write it as F transpose and then another vector x_t here. Then we're going to have x_t is going to be a function of x_t minus 1. That vector there is going to be an F and a G. I will describe what those are in a second. Then I'm going to have another vector here with some distribution. In our case, we are going to have a normal distribution also for that one. In the case of the AR(P), we're going to have x_t to be y_t, y_t minus 1. It's a vector that has all these values of the y_t process. Then F is going to be a vector. It has to match the dimension of this vector. The first entry is going to be a one, and then I'm going to have zeros everywhere else. The w here is going to be a vector as well. The first component is going to be the Epsilon t. That we defined for the ARP process. Then every other entry is going to be a zero here. Again, the dimensions are going to match so that I get the right equations here. Then finally, my G matrix in this representation is going to be a very important matrix, the first row is going to contain the AR parameters, the AR coefficients. We have p of those. That's my first row. In this block, I'm going to have an identity matrix. It's going to have ones in the diagonal and zeros everywhere else. I'm going to have a one here, and then I want to have zeros everywhere else. In this portion, I'm going to have column vector here of zeros. This is my G matrix. Why is this G matrix important? This G matrix is going to be related to the characteristic polynomial, in particular, is going to be related to the reciprocal roots of the characteristic polynomial that we discussed before. The eigenvalues of this matrix correspond precisely to the reciprocal roots of the characteristic polynomial. We will think about that and write down another representation related to this process. But before we go there, I just want you to look at this equation and see that if you do the matrix operations that are described these two equations, you get back the form of your autoregressive process. The other thing is, again, this is called a state-space representation because you have two equations here. One, you can call it the observational level equation where you are relating your observed y's with some other model information here. Then there is another equation that has a Markovian structure here, where x_t is a function of x_t minus 1. This is why this is a state-space representation. One of the nice things about working with this representation is we can use some definitions that apply to dynamic linear models or state-space models, and one of those definitions is the so-called forecast function. The forecast function, we can define it in terms of, I'm going to use here the notation f_t h to denote that is a function f that depends on the time t that you're considering, and then you're looking at forecasting h steps ahead in your time series. If you have observations up to today and you want to look at what is the forecast function five days later, you will have h equals 5 there. It's just the expected value. We are going to think of this as the expected value of y_t plus h. Conditional on all the observations or all the information you have received up to time t. I'm going to write it just like this. Using the state-space representation, you can see that if I use the first equation and I think about the expected value of y_t plus h is going to be F transpose, and then I have the expected value of the vector x_t plus h in that case. I can think of just applying this, then I would have expected value of x_t plus h given y_1 up to t. But now when I look at the structure of x_t plus h, if I go to my second equation here, I can see that x_t plus h is going to be dependent on x_t plus h minus 1, and there is a G matrix here. I can write this in terms of the expected value of x_t plus h, which is just G, expected value of x_t plus h minus 1, and then I also have plus expected value of the w_t's. But because of the structure of the AR process that we defined, we said that all the Epsilon T's are independent normally distributed random variables center at zero. In this case, those are going to be all zero. I can write down this as F transpose G, and then I have the expected value of x_t plus h minus 1 given y_1 up to t. If I continue with this process all the way until I get to time t, I'm going to get a product of all these G matrices here, and because we are starting with this lag h, I'm going to have the product of that G matrix h times. I can write this down as F transpose G to the power of h, and then I'm going to have the expected value of, finally, I get up to here. This is simply is going to be just my x_t vector. I can write this down as F transpose G^h, and then I have just my x_t. Again, why do we care? Now we are going to make that connection with this matrix and the eigenstructure of this matrix. I said before, one of the features of this matrix is that the eigenstructure is related to the reciprocal roots of the characteristic polynomial. In particular, the eigenvalues of this matrix correspond to the reciprocal roots of the characteristic polynomial. If we are working with the case in which we have exactly p different roots. We have as many different roots as the order of the AR process. Let's say, p distinct. We can write down then G in terms of its eigendecomposition. I can write this down as E, a matrix Lambda here, E inverse. Here, Lambda is going to be a diagonal matrix, you just put the reciprocal roots, I'm going to call those Alpha 1 up to Alpha p. They are all different. You just put them in the diagonal and you can use any order you want. But the eigendecomposition, the eigenvectors, have to follow the order that you choose for the eigenvalues. Then what happens is, regardless of that, you're going to have a unique G. But here, the E is a matrix of eigenvectors. Again, why do we care? Well, if you look at what we have here, we have the power G to the power of h. Using that eigendecomposition, we can get to write this in this form. Whatever elements you have in the matrix of eigenvectors, they are now going to be functions of the reciprocal roots. The power that appears here, which is the number of steps ahead that you want to forecast in your time series for prediction, I'm just going to have the Alphas to the power of h. When I do this calculation, I can end up writing the forecast function just by doing that calculation as a sum from j equals 1 up to p of some constants. Those constants are going to be related to those E matrices but the important point is that what appears here is my Alpha to the power of h. What this means is I'm breaking this expected value of what I'm going to see in the future in terms of a function of the reciprocal roots of the characteristic polynomial. You can see that if the process is stable, is going to be stationary, all the moduli of my reciprocal roots are going to be below one. This is going to decay exponentially as a function of h. You're going to have something that decays exponentially. Depending on whether those reciprocal roots are real-valued or complex-valued, you're going to have behavior here that may be quasiperiodic for complex-valued roots or just non-quasiperiodic for the real valued roots. The other thing that matters is, if you're working with a stable process, are going to have moduli smaller than one. The contribution of each of the roots to these forecasts function is going to be dependent on how close that modulus of that reciprocal root is to one or minus one. For roots that have relatively large values of the modulus, then they are going to have more contribution in terms of what's going to happen in the future. This provides a way to interpret the AR process.
