Here are some more facts on testing. The first one is particularly important. It says that statistical significance is not the same thing as practical importance. What does that mean? Suppose in our example, the sample average of lead concentrations is just slightly above the health standard of 15 parts per billion. For example, the sample average could be 15.05 parts per billion. Now, that may not be very relevant. After all, it's essentially the same thing as 15, but it may turn out that a statistical test we just did is highly significant. How can that be? Well, a statistical significance simply says that we can be confident that the effect is there. It doesn't say how big the effect is. So, how can it be that a small difference such as going to 15.05 will result in a highly significant test? Well, the answer is the square-root law. A large sample size makes the standard error small because the sample size is in the denominator. And with a small standard error, we may quickly get a very significant test result. For that reason, it's helpful to complement the test with a confidence interval. In that case, we might get a confidence interval that, for example, goes from 15.02 all the way to 15.08, and that, in effect, tells us that we can be confident that we are not far away from the health limit. It turns out that there's a general connection between confidence intervals and tests. A 95% confidence interval contains all values for the null hypothesis that will not be rejected by a two-sided test at the 5% significance level. Now, that's a difficult statement to digest. The best way to think about this is in terms of the previous example, rejecting the null hypothesis of 15 is the same as saying 15 is not in the confidence interval. Finally, keep in mind that there are two ways that a test can result in a wrong decision. It may be that the null hypothesis is true but we erroneously reject it. This is called a type I error, or you may know it under the name false positive. The second type of error, which is called type II error occurs if the null hypothesis is not true, but we fail to reject it. Remember, the standard rule for rejecting a null hypothesis is when the p-value falls below 5%. If that's the rule, that means that the probability of a type I error is, at most, 5%.