>> We're now going to review some of the basic concepts from probability. We'll discuss expectations and variances, we'll discuss Bayes' theorem, and we'll also review some of the commonly used distributions from probability theory. These include the binomial and Poisson distributions as well as the normal and log normal distributions. First of all, I just want to remind all of us what's a cumulative distribution function is. A CDF, a cumulative distribution function is f of x, we're going to use f of x to denote the CDF and we define f of x to be equal to a probability that a random variable x is less than or equal to little x. Okay. We also, for discrete random variables, have what's called a probability mass function. Okay. And a probability mass function, which we'll denote with little p, it satisfies the following properties. P is greater than or equal to 0, and for all events, A, we have that the probability that x is in A, okay, is equal to the sum of p of x over all those outcomes x that are in the event A. Okay. The expected value of a discrete random variable, x, is then given to us by this over here. So, it's the sum of the possible values of the random variable x. These are the xi's, weighted by their probabilities, p of xi. So, that's the expected value of x. If I was to give you an example. Suppose, for example I tosses a dice. So, it takes on 6 possible values. Okay, 1, 2, 3, 4, 5, and 6. Okay. And it takes on each of these values with probability, so that's wp, with probability 1 6th, with probability 1 6th and all the way down 1 6th. So, in this case, for example, the probability that x is greater than or equal to 4 is equal to, well, it's 1 6th for 4, 1 6th for 5, and 1 6th for 6, so that's equal to 1 6th plus 1 6th plus 1 6th equals 1 half. Likewise, we can compute the expected value of x. In this case, it is equal to 1 6th times 1 plus 1 6th times 2, and so on, plus 1 6th times 6. And that comes out to be 3 and a half. Okay. So, we also have the variance of a random variable. It's defined as the expected value of x minus the expected value of x, all to be squared. And if you expand this quantity out, you can see that you'll also get this alternative representation, so that the variance of x is also equal to the expected value of x squared minus the expected value of x, all to be squared. Okay. So, there, a discrete round of variables, probability mass function, and so on. So, let's look at a couple of distributions. The first distribution I want to talk about is the binomial distribution. We say that a random variable x has a binomial distribution, and we write it as x tilde binomial, or bin n, p, if the probability that x is equal to r, is equal to n choose r times p to the r by 1 minus p to the n minus r. And for those of you who have forgotten, n choose r is equal to n factorial divided by r factorial times n minus r factorial. So, the binomial distribution arises, for example, in the following situation. Suppose we toss a coin n times, and we count the number of heads. Well then, the total number of heads has a binomial distribution and we're assuming here that these are independent coin tosses so that the result of one coin toss has no impact or influence on the, the outcome of other coin tosses. The mean and variance of the binomial distribution are given to you by these quantities here. So, the expected value of x equals np, the variance of x equals np times 1 minus p. Now, there's actually an interesting application of the binomial distribution to finance. And it actually arises in the context of analyzing fund manager performance. We'll actually return to this example later in the course. But let me just give you a little flavor of it now. So, suppose, for example, a fund manager outperforms the market in any given year, with probability p. And that she underperforms the market at probability 1 minus p. So, we're assuming here that the fund manager either outperforms or underperforms the market, only two possible outcomes. And that they occur with probabilities p and 1 minus p respectively. Suppose this fund manager has a track record of ten years, and that she has outperformed the market in eight of these ten years. Moreover, let's assume that the performance, the fund manager performance in any one year is independent of the performance in other years. So, a question that many of us would like to ask is the following. How likely is a track record as good as this outperforming eight years out of ten, if the fund manager had no skill? And, of course, if the fund manager had no skill, we could assume maybe that p is equal 1 half. Okay. So, actually, we can answer this question using the binomial model, or the binomial distribution. So, let x be the number of outperforming years. Since the fund manager has no skill, then there are ten years, and the total number of outperforming years x, is then binomial, with n equals 10, 10 years, and p equals a half, okay? So, we can then compute the probability that the phone manager does at least as well as outperforming in eight years out of ten, by calculating the probability that X is greater than or equal to 8. So, what we're doing here is calculating the probability that the fund manager would have 8, 9, or 10 years out of 10 in which she outperformed the market. And that is given to us by the sum of these binomial probabilities here. So, these were the original binomial probabilities on each slide, and we summed them from r equals 8, to n. And n, in this case, of course, is 10, okay? So, that's one way to try and evaluate whether the fund manager has just been lucky or not. One can compute this probability and if it's very small, then you might conclude that the fund manager was not lucky and that she had some skill. But actually, this opens up a whole can of worms. There are a lot of other related questions that are very interesting. Suppose there are M fund managers, how well should the best one do over the ten-year period if none of them had any skill? So, in this case, you don't have just one fund manager as we had in this example so far, we now have M of them, okay? And it stands to reason that even if none of them had any skill, then as M gets large, you would expect at least one of them or even a few of them to do very well. Well, how can you analyze that? Again, you can use the binomial model and what are called order statistics of the binomial model to do this. And we'll actually return to this question later in the course. Okay. So, let's talk about another distribution that often arises in finance and financial engineering, that is the Poisson distribution. We say, that x has a Poisson lambda distribution so lambda is the parameter of the distribution. If the probability that x equals r is equal to the lambda to the power of r times e to the minus lambda, divided by r factorial. And for those who have forgotten factorials, I also used it in the binomial model a while ago. R factorial is equal to r times r minus 1 times r minus 2, all the way down to 2 times 1. Okay. So, this is the Poisson distribution. The expected value and the variance of a Poisson random variable are identical and equal to lambda. So, for example, we'll actually just show this result here. It's very simple and the mean is calculated as follows. We know that the expected value of x is equal to the sum of the possible values of x, so these are the r's, times the probability that x is equal to r and r runs from 0 to infinity. We can calculate that as follows. So, we have the summation of r and the probability that X equals r. We know from up here, okay, and we can substitute that down in here and now, we just evaluate the sum. The first thing to notice is that when r equals 0, this term in the sum is equal to 0. So, we can actually ignore the 0, the first element, the 0 element and replace the summation running from r equals 1. So then, we get this quantity here. We can cancel this r out with the first r up here and write, this is r minus 1 factorial. We can also pull one of these lambdas out here leaving us with a lambda to the r minus 1. And now, if we look at this quantity here, this summation here, we see that this is the same as changing this to run from r equals 1 to r equals 0 and replacing r minus 1 with r and r minus 1 factorial with r factorial here. This total we see is equal to the sum of the probabilities. These are the probability that x equals r, so this is the sum of the probabilities that x equals 0, x equals 1, x equals 2, so this is equal to 1. The total sum of probabilities must be equal to 1, so this is equal to lambda. Okay, let's talk a little bit now about Bayes' theorem. Let A and B be two events for which the probability of B is nonzero, then the probability if A given B, and this is notation we'll use throughout the course, this vertical line means it's a conditional probability. S,o it's the probability of A given that B has occurred, well, this is equal to the probability of A intersection B divided by the probability of B. Alternatively, we can actually write this, this numerator probably of A intersection B, as being the probability of B given A by the probability of A. So, this is another way to write a Bayes' theorem. And finally, if we like, we can actually expand the denominator here, the probability of B, and write it as the summation of the probability of B given Aj, by the probability of Aj. Let me sum over all Aj's. For the Aj's, form a partition of the sample-space. What do I mean by partition? Well, I mean the following. So, Ai intersection Aj is equal to the null set, for i not equal to j, and at least 1 Ai, at least, at least one Ai must occur. And, in fact, because Ai intersection Aj is equal to the null set, for i not equal to j, I can actually replace this condition with the following, exactly one Ai must occur. Okay. So, that's Bayes' theorem. Let's look at an example. So, here's an example where we're going to toss 2 fair 6-sided dice. So, Y1 is going to be the outcome of the first toss, and Y2 would be the outcome of the second toss. X is equal to the sum of the two, and that's what we plotted in the table here. So, for example, the 9 here comes from the 5 on the first toss and 4 on the second toss. So, 4 plus 5 equals 9. So, that's X equals Y1 plus Y2. So, the question we're interested in answering is the following. What is the probability of Y1 being greater than or equal to 4, given that x is greater than or equal to 8? Well, we can answer this using this guy here on the previous slide. So, this is equal to the probability that Y1 is greater than or equal to 4 and X is greater than or equal to 8, divided by the probability that X is greater than or equal to 8. Okay. So, how do we calculate these two quantities? Let's look at the numerator first of all. So, we need two events here. Y1 must be greater than or equal to 4 and X being greater than or equal to 8. Okay. So, the first event is clearly captured inside this box here, okay, because this corresponds to Y1 being greater than or equal to 4. So, all of these outcomes correspond to that event. The event that X is greater than or equal to 8 corresponds to this event or these outcomes. So therefore, the intersection of these two outcomes, where Y1 is greater than or equal to 4 and X is greater than or equal to 8, is this area here, which is very light, so let me do it a little bit darker. So, it's this area here. Now, each of these cells is equally probable and occurs at probability 1 over 36. There are a total of 3, 4, 7, plus 5, 12. So that's 12 cells here. So, the numerator occurs with probability 12 over 36. And the, the denominator, the probability that X is greater than or equal to 8, well, that's what we highlighted in the red here. And the probability of that occurring, well, there's 12 plus these 3 additional outcomes equals 15 outcomes. So, that's 15 over 36, and that is equal to 4 over 5. So, that's our application of, of Bayes' theorem. Okay. So, let me talk a little about continuous random variables. We say a continuous random variable x has a probability density function, or a PDF, f. If f of x is greater or equal to 0, and for all events, A, the probability that x is in A, or the probability that A has occurred is the integral of the density, f of y, dy over A. The CDF, cumulative distribution function, and the PDF are related as follows, f of x is equal to the integral from minus infinity to little x of f of y dy. And, of course, that's because we know that f of x, by definition, is equal to the probability that X is less than or equal to x, so this, of course, is equal to the probability that minus infinity is less than or equal to X, is less than or equal to little x. So, this is our event A here and this definition here. So, A is now integrated from, A is now the event minus infinity less than or equal to the random variable x, less than or equal to little x, so that's what we have over here. So, it's often convenient to recognize the following, that the probability that x is in this little integral here, x minus epsilon of 2 and x plus epsilon over 2. Well, that's equal to this integral, x minus epsilon over 2 to x plus epsilon over 2 times f of y dy, okay? And if you like, we can draw, something like this. So, this could be the density, f of x. This is x here, maybe we've got some point here which is little x, and this is x minus epsilon over 2. This is x plus epsilon over 2. So, in fact, what we're saying is that the probability is this shaded area, and it's roughly equal to this value, which is f of x times epsilon, which is the width of this interval here, okay? And, of course, the approximation clearly works much better as epsilon gets very small. Okay. So, there are continuous random variables. Let me talk briefly about the normal distribution. We say that X has a normal distribution or write X tilde N mu sigma squared if it has this density function here. So, f of x equals 1 over root 2 pi sigma squared times the exponential of minus x minus mu, all to be squared divided by 2 sigma squared. The mean and variance are given to us by mu and sigma squared respectively. So, the normal distributions are very important distribution in practice, its mean is at mu, its mode, the highest point in the density is also at mu and approximately 95% of the probability actually lies within plus or minus 2 standard deviations of the mean. So, this is approximately equal to 95% for a normal distribution. Okay. So, this is a very famous distribution. It arises an awful lot in finance. It certainly has its weaknesses and we'll discuss some of them as well later in the course. A related distribution is the log-normal distribution. And we will write that x has got a log-normal distribution with parameters mu and sigma squared if the log of x, is normally distributed with mean mu and variance sigma squared. The mean and variance of the log-normal distribution as given to us by these two quantities here, and again, the log-normal distribution plays a very important role in financial applications.
