The course is Critical Thinking for the Information Age. I'm Richard Nisbett, this is lesson two, The Law of Large Numbers. This lesson deals with a reasoning principle that you've used all your life, namely, the law of large numbers. But you don't use it as much as you should. So I want to start with a simple question. I'll tell you that there's a city with two hospitals. One, with about 15 births per day and one with about 45 births per day. Now, as you know, there are about 50% of babies born who were boys and 50% who were girls. But which of these hospitals, do you think has more days when 60% or more of the babies born are boys -- the hospital with 15 births per day or the hospital with 45 births per day? I'll let you think about that for just a minute. I'll tell you that most people think there would be no difference. That there would be the same number of days in a year at the 15 birth per day hospital and the 45 birth per day hospital, when 60% or more of the babies born are boys. Well, let's do a simulation. This is the hospital with 15 births per day. Okay, so there's our 15 births. There's nine boys and six girls, so that's 60% boys. Does that seem at all unusual? Well, no, not really, as a matter of fact, if one of those boy babies were a girl baby, it would be eight boys and seven girls and you can't come any closer to 50, 50 than that. So it's not at all unusual. Now let's do a simulation of the babies born at the hospital with 45 births per day. Well, that's 27 boys born and 18 girls born, that's 60% boys. Does this seem unusual to you? Does this seem weirdly too many boys? Well, it should. Suppose, somebody said, let's do some coin flips for money, a buck each. If it comes up heads, my coin, then I win that trial, if it comes up tails, you win, and then this was the distribution, 27 heads and 18 tails. You probably would feel you'd been taken for a ride. So it is, in fact, an unusual pattern to get. Given that we know that there's a 50/50 ratio of boys and girls. You can ask how likely would it be that you were drawing from that kind of distribution, a 50/50 distribution, if you got 27 boys and 18 girls. And the answer is that's only going to happen 3 in 100 times. It could happen, but it is pretty darned unlikely. Now let's look at a simulation for 450 babies. That's about the number of babies born in the state of Michigan on a given day. I assume that seems very, very strange to you, to have such a remarkable number of boys and such a remarkably small number of girls, but, in fact, it's 60% again. It's 270 boys and 180 girls, so it's at 60% figure. How likely is it that you could get A pattern like this, given that the true proportion is 50/50. The answer is you could get something this extreme, not even one time in a million. Now, what's going on here? The principal that we use to understand this is the law of large numbers, which says that, sample values, for example, proportions, resemble population values as a function of their size. The larger the sample, the less likely it is that you will get a fluke, a very unrepresentative value. So 60% is a very unrepresentative value for 50%, which is close to the true value. But it's common to get that kind of difference with a small sample. If the sample gets large enough it becomes virtually impossible. The other kinds of population values for which the law of large numbers holds is mean, median, standard deviation. Anything that you can use to describe a sample and to describe a population is going to get more accurate the larger the sample. So you're going to get 60% or more boys at a hospital with 15 births every few days. You'll get 60% or more boys at a hospital with 45 births, maybe nine or ten times a year. And for the state of Michigan, you'll get a day when 60% of the babies born are boys, precisely never. Suppose I told you I knew an executive who needed to hire a manager. He interviewed a man with a great record, a terrific recommendations from his previous employers, but in the interview the fellow didn't have much interesting to say about my friend's company. Seemed to have low energy. So my friend told his colleagues that the guy shouldn't be pursued, just isn't for us, doesn't cut it. That sounds like the kind of thing that happens all the time, right? But is the executive's judgement really a reasonable one? To help you think about that, suppose I told you I know a soccer coach. For the non-Americans, I'll point out that that's the game you call football. So this coach is looking for a striker, and he goes to a practice for a high school kid, who has a great scoring record and terrific reviews from his coaches. But at this practice, the guy misses some easy points and he just doesn't seem in control of the ball. So my friend tells his colleagues that the kid shouldn't be pursued, he just hasn't got it. Now, is the coach's judgment reasonable or not? People who know sports are quite likely to say no that's really not so reasonable. One practice, it's just not that much evidence, there's lots of variability. I mean, you could have an off day and look much worse than usual, or everything can be hitting on all eight cylinders and you can look much better than usual. And this kind of variability is captured in an expression like, on any given Sunday, any team in the National Football League can defeat any other team in the National Football League. So let's go back to my friend, the executive. Do you still think he's reasonable? Well, the 30 minute unstructured interview is not that much evidence. In fact, people have looked at how well you can predict performance in college, medical school, Peace Corps, army officer training school and many, many businesses and professions, to see how well that interview rating predicts performance. And the correlation almost never exceeds 0.10, that's very, very small. That's equivalent to increasing the likelihood of hiring the better of two candidates from 50/50, which is what you would get if you were going to flip a coin to make the decision, to a 53% chance. Hardly seems worth the effort. If you have past performance and other judgments by other people, you can do quite well in predicting these same kinds of performance that I just mentioned. In fact, if you weight things properly that are in the folder, you can raise the chances of picking the right person to 65% or 75%. So this is the principle of law of large numbers, again, but I'm going to modify it slightly for these purposes, to say that sample values for events having a chance component resemble population values for those events as a function of their size. In fact, the law of large numbers only applies where there is some kind of a chance component. There's no chance component to your measurement of how far it is between London and Paris, and the law of large numbers is irrelevant there. One measurement will do you, but sports are very variable. We know that performance can be much higher or much lower on any given occasion. But most people never observe all that many interviews. And you don't get to see how well the prediction from the interview corresponds to performance at the person on the job. The truth is the employer's judgment is even worse than the coach's judgement because interviews are not a sample of job performance or school performance. They're a sample of interview performance. My daughter used to be an opera singer and she told me that auditions and performances require different skills. Interviews and performance on the job require different skills. So the law of large numbers applies to all kinds of events in everyday life. We apply the law of large numbers when we see the variability, the error, but not for events that are just as important where we don't. The principle, here, is if your variable is human behavior, assume there is error variance, and adjust your judgement accordingly.
