Here's a different question about colors and M&Ms. Is the color distribution the same for milk chocolate M&Ms, peanut M&Ms, and caramel M&Ms? So in this case, we would have to compare several populations. That is called a test of homogeneity. The null distribution is that the color distribution is the same for all of the populations, which in this case, are milk, peanut, and caramel. The chi-square test of homogeneity assumes that the samples are drawn independently for each population, and also independently across the populations. To see how the mechanics of the test work, let's look again at the survival data for the Titanic. The first thing to note is that we're not sampling from a population. The data we have are not a random sample of the people on board, rather the data represent the whole population. So, this is not the usual setup where we sample from a population, but it may still meet the assumption that we draw independently. The way to think about this is that the chance process resulting in survival or death, is the result of a sequence of random events that occurs when people try to find their way out of the ship, get into a lifeboat, or have to jump into the water, and whether they are being rescued out of the water in time. So, we can think about the 325 observations on first-class passengers as 325 independent draws from a probability histogram that gives a certain chance for survival. Likewise, the 285 observations about second-class passengers are drawn from a probability histogram for second-class passengers, which may be different. Now, the null hypothesis says that the probability of survival is the same for all four probability histograms. And that's the null hypothesis we're going to test. The chi-square test in this situation is quite similar to the one before. That means we compare the observed counts to expected counts. We already have a table of observed counts. So next, we have to find the table of expected counts. The way this is done is that realizing that under the null hypothesis, the probability of survival is the same for all four populations. That means we can estimate the probability by simply pooling all the data. In total, there were 713 survivors among the 2,229 people on board. So, we estimate the probability of survival as 713 over 2,229, which is 32%. Then the expected number of surviving first-class passengers is that 32% times the number of first-class passengers, and that comes out to be 104. We repeat this calculation for the other seven categories and that gives us and our table of expected values. Then we compute the chi-square statistic just as before. That is, we look at the difference between observed and expected, square it, divide it by expected, and sum over all eight cells. In that case, the chi-square statistic gives us 192.2. Again, we have to compare this number to a chi-square distribution but the degrees of freedom in this case, it's different. The degrees of freedom is given by the number of columns - 1, times the number of rows - 1. So, we had four ticket classes, so we had four columns. So, 4 - (-3), and then we had two rows, 2 - 1 = 1, so the degrees of freedom is the product of 3 and 1 which is 3. So then we have to compare the chi-square value of 192.2, against a chi-square distribution with three degrees of freedom. If you look at the picture, you see that this 192 is all the way out there and there's essentially no area to the right so the p-value is 0%, and that is very strong evidence against the null hypothesis. So the conclusion would be that the chances of survival depended on the ticket class. Finally, the third application of the chi-square test is testing independence. As an example, we may ask whether gender being male or female is related to voting preference. That means voting liberal or conservative. In this case, we have two categorical variables. One is gender and the other is voting preference. The null hypothesis is that the two variables are independent. The alternative hypothesis is that there is some kind of association. Again, the chi-square test can be used to test this null hypothesis. We sample data from the population by sampling people at random, we record the outcomes for the two categorical variables, and that gives us columns in a two-by-two table. It turns out that the chi-square statistic and p-value are computed exactly as in the case of testing homogeneity. That means we also used the same way of computing the degrees of freedom. Since the chi-square test works the same way both for testing independence and testing homogeneity, it's easy to mix those up. Here's a table that compares what's different between these two cases. The first difference is in the sampling. The chi-square test of homogeneity measures a single categorical variable on several samples that were obtained from several populations. In contrast, for the chi-square test of independence, we have only one sample, but we measure two categorical variables on the sample. Also, the research questions are different. The chi-square test of homogeneity tests whether the different groups are homogeneous, which means that they have the same distribution of the categorical variable. In contrast, the chi-square test of independence checks whether the two categorical variables are independent.
