This week, we'll learn about the Gaussian distribution for statistical modeling in robotics. Gaussian distribution is a widely used continuous probabilistic representation, and it provides a useful way to estimate uncertainty in the world. We'll start by discussing the one-dimensional Gaussian distribution, and then move on to the multivariate Gaussian. Finally, we'll extend the concept to models that use mixtures of Gaussians. In this lecture, let's learn how we can express uncertainty with one-dimensional Gaussians. Before we talk about the details, it is important to understand why we learn about the Gaussian distribution. What makes the Gaussian distribution useful and important? First, only two parameters are needed to specify the Gaussian. They are called the mean and variance, and they capture the essense of the distribution. They are also easy to compute and interpret. Second, mathematically the distribution has some nice properties. For example, product of Gaussian distributions forms another Gaussian. So you don't need to worry about encountering other forms of distributions when you perform operations on the Gaussian model. Lastly and more theoretically, the central limit theorem tells us that the expectation of the mean of any random variable converges to the Gaussian distribution. That implies Gaussian is a proper choice for modeling noise and uncertainty. Because of both practical and theoretical benefits, we use the Gaussian distribution. As an example, let's look at an image processing problem. We'll see how the Gaussian distribution can be used to model a target color. Here is a view from a soccer-playing robot's head-mounted camera. Clearly there are two balls in the image. One is red and another is yellow. The robot wishes to detect the yellow ball so that it can kick it. It is trivial for humans to make this distinction. But robots can find it difficult to map raw pixel values into colors like red and yellow. What robots need to detect the ball is a color model that represents redness or yellowness. To this end, let's look into the pixels of only the yellow ball in the image. We can segment out the yellow ball like this and really inspect the hue of these pixels in this example. Hue is a component of the HSV representation of colors. Let's plot a histogram of the yellow ball pixels based on the hue value. As you can see, but here is not a single value. Instead, the distribution of values is centered at around 53 and spreads out to a certain extent. If we naively use all the values within the histogram, it will require a lot of memory, as many memory as the number of pixels. One succinct way of capturing this center and spread is to use a Gaussian model. Now we're going to try to understand the mathematical expression and the parameters of the Gaussian model. After that, let's come back to this example again. Gaussian distribution is expressed as an exponential term multiplied by a scalar. We want to know the probability that x, the variable, lies within our Gaussian distribution. We call this probability density function. We use p(x) to write this. Mu is the mean of our Gaussian and sigma is its standard deviation. When it's squared, we call it variance. Mu and sigma are the two model parameters we described earlier in the first slide. In our example of the colored ball x is the hue value of a sampled pixel. p(x) is the probability that this sampled pixel belongs to a yellow ball given the mean and variance of our hue model of the yellow ball, which we are going to estimate later. I think everyone likes a picture. Let's think about what Gaussian would look like. We'll first consider when a distribution has zero mean and univariance. This is a often called the standard normal distribution. If you'll look at the graph and look inside the exponential parts of the expression, you will see this distribution is symmetric about the mean, which is 0 in this case. Also you should notice that the value of p(x) gets very small as x goes far from the mean. This is due to the minus sign inside the exponential function. The last thing to notice is the scalar term outside the exponential function. Remember Gaussian is a probability distribution and thus its integral, the integral of p(x), must be 1. Now let's consider other cases with different values for the mean. The gray curve is a standard Gaussian curve. Compared to this, when the mean is -1, the curve is shifted to the left by 1. If the mean is 1, the graph is shifted to the right by 1. The mean value determines the center of the distribution. We can also say that the peak location of the distribution changes. Critically, the actual shape has not been changed, only shifted. The variance changes the spread of the distribution. If the variance increases to 2, the curve spreads out as compared to the standard Gaussian curve. Also, the peak value decreases so that the integral is still 1, which fulfills the properties of a probability density function. Conversely, a smaller variance tightens the curve and the peak value becomes bigger as well. So that the integral remains 1. We have seen the two parameters of the Gaussian distribution. The mean mu represents the center of the distribution. And the variance sigma squared represents the spread of the distribution. Now that we have understood the two parameters of the Gaussian, let's get back to the ball color example and try to apply what we've learned to represent the ball color. Instead of having all the data points in this histogram, we can find a Gaussian curve with some mean and variance that approximates the sampled distribution using only two numbers. The mean here is roughly the peak of the histogram, while the variance signifies the spread of the sample distribution. Having a larger variance implies a larger uncertainty of what hue values are likely to be yellow. On the other hand, If we had a small variance, you'd be more certain of what yellow actually is. We are going to talk about how to estimate the parameters from data in the next lecture.
