0:00

Hello, everyone.

Â In this lecture, we will try to estimate model parameters of

Â autoregressive processes of order 2 doing some simulations.

Â In others words, we are going to simulate an AR(2) process.

Â So, the objective is to estimate the variance

Â of the white noise in the AR(2) process.

Â And estimate the coefficient of the simulated AR(2)

Â process using the Yule-Walker equations in a matrix form.

Â AR(2) process with mean zero would be in this format without any constant,

Â and the Zt's are innovations, the white noise.

Â And we tried to simulate this process for phi 1,

Â the first coefficient being 1 over 3.

Â Phi2, the second coefficient is 1 over 2.

Â And sigma is 4.

Â In other words, this AR(2) model has three parameters.

Â So we're going to use the Yule-Walker equations.

Â And eventually Yule-Walker estimators to actually estimate each of

Â these coefficients, the phi1 and phi2 and also sigma in this problem.

Â 1:12

So we estimate the coefficients of the model by first finding r1, r2.

Â Remember r1, r2 are sample auto correlation function.

Â We're going to use acf routine in r.

Â And we're going to solve the system using the following

Â matrix form of the Yule-Walker equations.

Â This is our r matrix which is symmetric, it has an inverse.

Â And then we're going to find the inverse of it and

Â multiply it to the left side which is r1, r2.

Â And we will get phi1.hat, phi2.hat.

Â But before we start the estimation, let me mention something about the sigma.

Â So, we know how to estimate the coefficient in the AR(2) model but

Â how about the sigma?

Â That's also another parameter of the system so

Â we should be able to estimate that as well.

Â So let me note the following.

Â If you look at this system, we assume that before we start with the Yule-Walker

Â equation, we already assume that it is a stationary AR(2) process.

Â So this is a stationary AR(2) model, AR(2) process.

Â We take the variants from both sides.

Â We get the variants of Xt, variants of Xt minus 1, variants of Xt minus 2.

Â Which all of them are the same because we have weak stationary or

Â covariant stationary process.

Â But since Xt minus 1, Xt minus 2 have some correlation, we also have

Â this covariance Xt minu 1, Xt minus2 term here and the variance of Zt is sigma.

Â So we're going to use this equation to get the sigma.

Â So sigma will be equal to the variance of this process

Â is actually gamma 0 autocovariance at lag.

Â And then we have 1 minus phi1 squared phi2 squared.

Â But if I pull out variance from here this covariance,

Â this is of the autocovariance at lag1, this is gamma 1.

Â If you plot variance you will get gamma 1 over gamma 0 which is actually rho 1.

Â So this is basically how we can estimate sigma.

Â But we can actually simplify this a little bit more.

Â Realize the following, from the Yule-Walker equations in the matrix form,

Â we realized that rho 1 is actually from the matrix multiplication is equal to

Â phi1 plus rho1, phi 2.

Â In a similar way, rho2, is the same as phi1, rho1 plus phi2.

Â This comes from the Yule-Walker equations.

Â Then, I take this expression which is inside this sigma expression.

Â Sigma is equal to gamma 0 times that.

Â Now here, we can separate this 2, phi1 phi2, rho1 expression into two of itself.

Â Of phi1 phi2 rho1, phi1 phi2 rho1.

Â In these two terms, we pull out phi1.

Â In these two terms we pull up phi2 and

Â we realize that the parenthesis is actually rho1 and rho 2.

Â In other words, we actually get sigma.

Â We get a formula for sigma square.

Â Which is gamma 0 autocovariance of the system at lag0.

Â The variance of Xt times 1 minus phi1 rho minus phi2 rho2.

Â Now, we can estimate sigma from here.

Â And this is going to be called the Yule-Walker estimator, instead of gamma 0,

Â we're going to get C0.

Â This is a sample of autocovariances at lag0.

Â We are going to use ACF routine to get this.

Â 5:00

So let me mention the following.

Â Now I'm going into the details of the simulation.

Â And I'm going to open up the code.

Â Code is written in Notebook.

Â So at this point, if you haven't all ready, stop the video.

Â Go and open up the Notebook.

Â Which is called AR (2) simulation.

Â And we're going to carry out every step that I'm talking about in this

Â presentation.

Â Actually it's written already in that notebook.

Â And try to run every code block in Notebook.

Â So, let me first go over this and then I'll open it up as well.

Â Number of data points, we're going to choose 10,000 data points.

Â And we're going to use a few routines here arima.sim, this is the simulation.

Â So arima.sim simulates data from the arima models.

Â Well, what is arima?

Â This is auto regressive integrated moving average models.

Â We have about moving average models.

Â We have talked about auto regressive models.

Â But we haven't connected them yet but

Â we will still use this model to get to the AR model.

Â We're going to use plot routine for plotting.

Â We're going to use acf() to find out the correlation function.

Â We will also use that to find the autocovariance function.

Â And we have done this before.

Â Matrix, this basically defines the matrix with dimension m and n.

Â And we're going to use solve(R,b).

Â Where R is a matrix, b is a vector.

Â And this routine solves Rx=b and gives a solution.

Â 6:37

Sigma, we're going to choose 4.

Â And phi[1:2], this is phi1 and phi2.

Â It is defined with this array, 1 over 3, 1 over 2.

Â N is 10,000.

Â We will set.seed(2017),

Â so that we will both get the same dataset.

Â So you can reproduce exactly what I'm reproducing in this lecture.

Â And ar.process, we are going to use arima simulation.

Â This is the model, model takes the list.

Â And then if you write ar, this specifies the coefficients of the ar model.

Â And standard deviation is equal to 4.

Â Basically specifies standard deviation of the innovations, the random noise.

Â And if you do that, since you have set the seed(2017),

Â you should get the exact same ar.process.

Â And if you, for example, look at the first five elements

Â out of 10,000 points in this time series, you should get the following numbers.

Â 7:37

Then, what do I need?

Â I need r's in my Yule-Walker equations.

Â I'm going to get my r's [1,2] or R1 and R2.

Â ACF gives you sample auto correlation function for a few lags.

Â If I take this $acf which means I'm going to the acf part only.

Â And I'm going to look at acf2 and acf3,

Â because the first element in this array is actually 1.

Â That's actually row 0.

Â Okay.

Â So if you look at this, r[1] and r[2] are going to be following numbers.

Â I'm going to define R as a matrix, [1,2,2].

Â What does it mean?

Â We want to have a 2 by 2 matrix, with all elements 1.

Â So we get this R.

Â But, how do I do that so that I can actually edit this?

Â So I'm going to edit 1, 2, this term and this term, by r1.

Â This is the matrix R in AR(2) process.

Â Let's define matrix b, which is the right hand side which is r1 and r2.

Â So we put them side by side.

Â 8:44

And we try to solve (R,b).

Â Because R now we have it, b now that we have it.

Â We can solve it.

Â Once we solve it in R, R gives us the following answers.

Â The first one is an estimate for phi1.

Â The second one is an estimate for phi2.

Â So we called them phi1.hat and phi2.hat.

Â And we can actually put them together here.

Â This is an array to see phi1 and phi2.hat.

Â And I define this as a matrix, 2 by 1 matrix.

Â And then phi1.hat becomes that matrix.

Â 9:18

Just define c0 as an autocovariance at lag0.

Â That's acf1, because the index of R always starts at 1.

Â So we have to do acf1 here.

Â And then type to do covariance so that it doesn't find out the correlation function,

Â it finds out the covariance function.

Â And we calculate the var.hat.

Â Var is for variance, so we are trying to estimate the variance.

Â Well, we just did it.

Â This is c0 times 1 minus phi1 r1, phi2 r2.

Â Basically the product of phi1.hat with r.

Â And then we're going to partition our screen,

Â our output device, into three rows and one column.

Â In the first row we plot the process with this title.

Â In the second row, we're going to have acf, autocorrelation function.

Â And in the third row, we're going to have a partial autocorrelation function.

Â Okay, so this is the Notebook.

Â 10:21

We have the name, which is Yule-Walker Estimation- AR(2) Simulation.

Â This is the name of the file Simulation of AR(2) process.

Â We are simulating AR(2) process.

Â Let's set seed the common number, so that we can produce the same dataset.

Â So let me run this cell, okay?

Â Now, let's look at the model parameters, we're going to say sigma is 4.

Â And this is phi equal to null.

Â Now let's define phi, phi is basically 1 over 3 1 over 2, phi1 and phi2.

Â Let's just print it and just to make sure to check that we have the right phi.

Â Exactly this is 1 over 3, this is 1 over 2.

Â Okay, let's run.

Â And n equals 10,000.

Â All right and we come to a simulation.

Â We simulate using arima simulation.

Â First entry in this function is n, which is 10,000, right?

Â So we are simulating a time series with 10,000 points, what is the model?

Â Model has to be a list.

Â We have to give the AR,

Â in author arrays of coefficients which is 1 over 3, 1 over 2.

Â We can actually write here phi1, phi2 if you like.

Â And we have to make sure that the standard deviation of the innovations,

Â the random noise is 4.

Â And the standard deviation is equals to 4.

Â And let's make sure we have that right AR process.

Â Let's run this and we get the first five entries in the time series.

Â The first five data points.

Â 11:59

In other words, R is null, we define R.

Â R1,2 is going to be ACF.

Â We do not want to plot so we say plot is false.

Â But we have to take second and third term because the first term is actually rho0.

Â So first term is always 1.

Â We want to get r1 to r2,

Â we have to look at the second and third entries in the ACF.

Â Okay, if we do that, we get r1 here, this is r1.

Â Autocorrelation coefficient in lag1, autocorrelation coefficient at lag2.

Â And let's define matrix R.

Â 12:37

So matrix R is going to be a two by two matrix, every element is 1.

Â Here we go.

Â This is our r, every element is 1.

Â And if I look at this cell, I am editing first row,

Â second column and second row, first column.

Â This other not the main diagonal but other diagonal by r1.

Â So if you do that, run the cell.

Â We will update the following R.

Â This is exactly what we were looking for.

Â This is the R that's coming from the Yule-Walker equation.

Â Let's define R matrix b which is on the left-hand side.

Â And that's basically r1 and r2.

Â Okay, now its time to find the coefficients, phi1.hat phi2.hat.

Â All we have to do, we have to solve Rx=b and how do we do this?

Â We say, solve(R, b), right?

Â Solve(R, b) will give us a phi1 and phi2 together.

Â But if I look at the first element, that's phi1.

Â If I look at the second element of that matrix, that is phi2.

Â And I take this phi1 and phi2 and put it into the matrix, phi1.hat.

Â This is matrix with two rows and one column.

Â And if we do that we get the following.

Â So the first estimate here is for phi1.hat.

Â This is phi2.hat.

Â Okay.

Â So we're almost done, right?

Â We have the AR(2) process.

Â We have phi1 and phi2.

Â Note the following, the original phi1 was 0.333.

Â This is 0.34 close, but not really close.

Â And here we have phi2.

Â The original phi2 is 0.5.

Â Estimation gives us 0.48, not bad.

Â We can get a better estimate by increasing the number of data points.

Â 14:30

Let's look at the variance because this is the only thing that's remaining,

Â is to estimate the variance of the innovations.

Â So we take the autocorrelation function, but we say type covariance, right?

Â We will get autocovariance at lag0, and

Â we have to start from 1 because of the indexing in the r.

Â And then rho.hat is defined to be autocovariance, this is gamma 0,

Â estimation of the gamma 0 c0 times 1 minus that product of our coefficients and

Â our autocorrelation coefficients.

Â And then we estimate this we get var.hat which is the variance,

Â the estimation of the variance which is 16.37.

Â Remember the original variance was 16, the square of 4, What we get here is 16.37.

Â And if we plot, so if we partition the output into three rows,

Â one column and we plot the time series acf and pacf.

Â And if we do that we obtain the following plot.

Â This is our time series.

Â This is the ACF and this is a PACF.

Â Just note the following, this abuses the time plot, but if you look at the ACF.

Â The ACF is decaying, eventually, right?

Â This is very much like the typical AR process.

Â And if you look at the partial ACF, partial autocorrelation function,

Â there are only two significant partial autocorrelation coeffitents at lag1,

Â at lag2.

Â And then as if there's no autocorrelation, that is very typical.

Â Because, PACF of ARP process has to cut from lagP.

Â Here we are looking at the AR(2) process.

Â So we shouldn't expect to see anything after lag2.

Â So what are the results?

Â Result is that phi1 is estimated by 0.34,

Â phi2 is estimated by 0.48, sigma is estimated by 16.37.

Â This was the actual model.

Â And this is the fitted model.

Â