Welcome back to Practical Time Series Analysis. We're looking at stochastic processes and their realizations called time series. And in these lectures, we're looking at them through the lens of stationarity. Stationarity is a crucial concept for us and it's a very important idea that allows us to try to say something meaningful about the stochastic process, a complicated mathematical object based upon a single realization or a time series. Perhaps a day that's set that you have acquired. This is something you can't do with a coin. If you have a coin and you observe tails on one toss, you can't really say anything meaningful about the coin or at least the distribution of heads and tails. All you can really say is that yes, this coin can give a tails but you can't say anything beyond that. So, stationarity really helps us to get some good work done. We're looking at stationarity through some very simple examples as we get started. These are more mathematically oriented. And as we move through the course, we move more into data sets. Right now, we're thinking about white noise. White noise will be trivially stationary. Random walks, which will not be stationary. And we'll look at an introduction to moving averages. These will be stationary processes. Recall the definition. Process is weakly stationary if the mean function as we look up and down the stochastic process and look at the average going on of each point, the mean function is constant. It is the same everywhere we look. The ACF, the autocovariance function, but depends just upon lag spacing. Again, it doesn't matter where you are along the process. If you have two random variables and you would like to know their covariance, all you need to know is how far away they're separated. Not where they are along the process. As promised, white noise is stationary. If you think of a random variable family, let's say a set, a sequence of IID random variables, they might be normally distributed but really they don't have to be. All we care about at the moment is that they're independent, identically distributed with mean of 0 and constant variance. Then the mean function, as a function of index t is 0 everywhere, so of course it's constant. If you look at the autocovariance function, gamma of t1 and t2, then we find that that's essentially a delta function, it's 0 when t1 and t2 do not agree. In other words, when you have two different random variables and as sigma squared, it reduces the variance when the subscripts agree. So, almost trivially you could say white noise is stationary. Random walks on the other hand are not stationary. Let's build a random walk off of a family of IID random variables. I'm using mu and sigma squared for the mean, and the variance for each one of the random variables. Mu could be 0, but in general, we'll go with a generic mu. We build a walk in t steps as your first position will be just where you got to off of your first variable. Your second position is where you get to by adding your first position and now taking another step of size to be determined by Z2. And we continue in that way, moving to the left or the right in random amounts. Your position at any time, t then, is just the sum, the aggregate of all the individual steps you took. A journey is really just the sum of its individual steps. When we explore the expected value as a function of index t here for our position x. Then and it can encourage enough to think about expected value, not really as a number associated with random variable but more as an operator that will make many variable manipulations much, much simpler to comprehend. Then we take an expected value of sum of these independent random variables. The expected value operator moves through the sum. That is an appropriate independence that just happens with random variables generically. And we find that the expected value of position looks like t times Mu. In other words if mu is not zero, the expected value is growing with time. Same for the variance. Since the X sub t is built on a family of independent increments here, the Z of t, then that will allow the variance upper to move through the sum. In general, it won't, now that there's a dependency structure among Z. But here, we started with independent identically distributed random variable. So, the variance operator moves to the summation, no problem. Variance grows with time. Variance is increasingly linearly with time. To have a meaningful process, we won't take sigma squared equals to zero. So, you are seeing that the variance is not constant. If the variance isn't constant, your process is not stationary. Another one of the canonical stochastic processes has to do with taking a family of random variables. We'll work with IID, independent identically distributed Z sub t. We'll give them zero mean and constant variance. We'll define a moving average process of order q as this called as X of t is equal to a linear combination of the underlying Z's. You can center your notation by looking at Z of t and then moving up and down along Z of t. But we'll follow the notation, the convention that says that X is a function of index t is equal to the noise at t plus the noise at t-1 and we're giving a certain weighting as we move through. There are different sets of beta that people like for different processes. You might have an image and you might be smoothing it or you might be doing edge detection. There are varieties of reasons people have for doing things like moving average processes. We're not making the claim that you see moving average processes just by themselves in nature all that often. It's a little bit hard to come up with an example of a naturally occurring moving average process just in its simple form like this. But the procedure of taking components and weighting them and adding together is really very basic, very common, and so it's important to study this. We'll also see a relationship later between moving average and auto-regressive processes that'll make this worthwhile. We can do some nice theoretical things with moving average processes to make our lives easier. We should look at a picture. White noise process up on top, no real structure to speak of, it's just noise. Now, down below, we've created a moving average process, where we let Q = 3. I did a simple moving average. So, we're just taking our components, adding them together, and dividing by the number of components. So, with Q = 3, we're dividing 4(Q+1) components where we're just taking an average of four components. You can see that we're losing some of our higher frequencies and gaining some low frequencies. What we're doing is seeing structure between neighbors. If your random variables are close together, there is actually going to be a dependency structure now. That's with Q = 3. I'm going to show you now and we hope that it'll just layover perfectly, Q = 9. So, let me move back. There's Q = 3. When I go to Q = 9, we induce still longer scale correlations, relationships between neighbors. We're smoothing even more, and I guess this just makes sense. We're including nine numbers, or ten actually, numbers in our average rather than nine. In this video, we've looked at some very basic examples of stochastic processes and we've studied their stationarity. White noise is stationary, perhaps trivially so. Random walks, even if there's zero mean, are not stationary. The variance grows with time. And we started looking at moving averages. In the next lecture, we'll actually explore the autocoveriance structure of the moving average process and look at its stationarity.