0:00

Let's assume that our sample is exactly normally distributed.

Â Whether this is because we're willing to stomach that assumption because we have

Â a large sample size and we're applying the central limit theorem or

Â if we're willing to just assume that the underline population is exactly

Â normally distributed.

Â Either way, for the sake of argument let's assume that X bar is normally distributed,

Â and that sigma, the standard deviation of the population is known.

Â 0:36

This is the same as just saying we're going to reject if x bar is bigger than 30

Â plus this z quantile times the standard err of the mean.

Â Now notice under the null hypothesis X bar follows this distribution.

Â Normal with a mean equal to mu naught.

Â In this case 30.

Â And the variance equal to sigma squared over N.

Â Under the alternative it follows the same distribution.

Â The only difference is instead of mu naught we have mu a.

Â Where mu a is the value under the alternative.

Â 1:19

Mu naught plus z times sigma over square root n or

Â larger, where this probability simply calculated with mu equal to mu a.

Â So here we have mean mu, mean equal to mu a, our standard deviation is

Â the standard error of the mean, and I've put lower.tail equals false.

Â So that I get the upper probability.

Â Now notice if I were to plug in mu equal to mu naught then I should get alpha, and

Â now as I plug as mu a moves away from alpha

Â the power should get larger and larger.

Â So let's try it.

Â 1:54

Suppose someone were to give me this information that they wanted to conduct

Â this study, they wanted to test whether or not mu was 30 for

Â this population or it was larger than 30.

Â So mu naught equals 30.

Â They were interested a difference as large as 32,

Â their end was 16,

Â that they were hoping to get, and they knew that sigma was around 4.

Â Okay?

Â So here I've plugged in the values mu naught equal 30, mu a equals 32,

Â sigma equals 4, n equals 16, here's my z is my normal quintile, and then first

Â I want to show you if I plug in mu equal to mu naught that it should give me 5%.

Â So here I plug in mu equal to mu naught and I get 5%.

Â Now I'm plugging in mu equal to mu a 32, and you see that this jumps up to 64%.

Â So, there's a 64% probability of detecting a mean as large as 32 or

Â larger if we conduct this experiment.

Â 2:58

So here I'm plotting the power curves,

Â which is the power as a function of Mu a,

Â as n varies by color here, and.

Â As we head to the right on this plot, so as we head along this axis,

Â that's mua getting bigger, and this axis is power.

Â Okay?

Â So let's take a specific one of these lines and look at it.

Â So this line, right here, is the power when n equals a.

Â And what you can see is all of the lines, is including the one that we're discussing

Â right now, converge at 0.05 as mu approaches 30.

Â And then what you can see is that power increases as mu a gets larger.

Â 3:51

Okay? And basically that means we're more likely

Â to detect a difference if the difference we want to detect is very big.

Â And that makes a lot of sense.

Â If something's a huge effect, we should be very probably,

Â it should be very probably to detect it.

Â And then the other thing I would note is that as we head up here,

Â we're seeing sample sizes doubled with each line, I start out with N equal to 8,

Â then I move to N equal to 16 right there and then 32 and then 64 and then 128.

Â And what you can see is the curves all getting shoved up to higher and

Â higher power earlier and earlier.

Â And this makes a lot of sense as well.

Â In other words, as we collect a lot more data, we should be more likely

Â to detect a power of a var, of a specific, for a specific value of mu a.

Â And so that's why, the mu n equal to 128 curve is uniformly above.

Â 5:01

Let's use our studio's manipulate function

Â to evaluate power as it relates to the two normal densities.

Â So here I'm going to do library manipulate,

Â then I'm going to define mu0 to be 30.

Â 5:16

Then I'm going to define a plotting function that depends on the population,

Â standard deviation, the mean under the alternative, the sample size,

Â and the type-1 error rate.

Â Then it does ggplot.

Â So, I'm, then it's going to execute that plot.

Â But it's going to give me a slider so that I can vary all of these parameters.

Â 5:55

Mu a was 32, n was 16 and alpha was 5%.

Â So what this plot is saying is

Â under the null hypothesis here's the distribution of our sample mean.

Â It's centered at 30 and it has a variance of sigma squared over n.

Â Under the alternative.

Â 6:23

We've set a critical value, so that if we get a sample mean that's

Â larger than a specific threshold we reject the null hypothesis.

Â That's this line.

Â We set this line such that the probability, if the red density is true,

Â 6:42

the null hypothesis is true this area,

Â the probability of getting statistics larger than it is 5%.

Â Now power is

Â nothing other than the probability of getting larger than this

Â line which is calibrated to have this area under the red curve is 5%.

Â The probability that we reject if in fact the blue curve is true.

Â That's the power.

Â Here's 1 minus the power or the type two error rate.

Â 7:15

Now let's start varianting and, and see what happens.

Â So if we move it so that its all the way down at 1% that's just saying that this

Â area right here under the red curve needs to be 1, calibrated to be 1%.

Â 7:44

This area is going down as that thing moves.

Â And what's this, what is this thing saying?

Â By moving alpha down we're making it harder to reject a null hypothesis.

Â We're making the requirement of having a lot more evidence.

Â Before we conclude the alternative is true.

Â That simply re, results in less power.

Â 8:29

If we increase alpha to the highest level I set here,

Â now we have a 10% type one error rate but we have much better power.

Â In other words, if we are willing to be a little bit looser in how much,

Â when we reject, if we, you know, get smaller means and we're still rejecting.

Â Then we'll get better power.

Â Of course we do have a larger type I error rate.

Â 10%, in this case, instead of 5%.

Â So let's set it back to 5%.

Â Let's see what happens as we decrease sigma.

Â Sigma goes all the way down to 1.

Â Okay. Now our black line moves.

Â Right. Because I've, I've,

Â these are not standard normal curves.

Â These are, so our rejection region isn't always so many standard errors from the,

Â from the mean in that, instead we've decided to plot this on the,

Â on the scale of the problem.

Â So this black line then depends on sigma.

Â So as we move it, as we move sigma, the black line moves with it.

Â Okay so let's move sigma down to the lowest low, level I'm a,

Â allowing which is sigma equal to 1 in this case.

Â Okay so our black line has moved down and

Â it's still calibrated so that this area is 5%.

Â But what we've seen is we have so

Â little variability in the sample mean that the probability of rejecting,

Â the probability of getting larger than the black line, if the blue distribution,

Â alternative distribution, is true, is almost 100%.

Â 10:06

Well, here sigma very large.

Â Again we're still calibrated, so that this area is 5%, but now power is much lower.

Â All this is saying, is that, if we have a lot more

Â noise in our measurements, then we're going to have lower power.

Â 10:47

And the black line doesn't depend on the mean under the alternative, so

Â it doesn't move, and then as it moves towards 30, you can see the power's

Â getting lower, and the lowest the power can be is if it's, lies right on top of

Â the null distribution and then this, the power'll be exactly point zero five.

Â Then it'll get a little bit bigger as it moves further away and further away and

Â further away, until if we're at 35 we have almost 100% power.

Â 11:21

So, remember what happens as n increases.

Â Our sample mean gets less variable,

Â more observations go into our sample mean it gets less variable.

Â So lets see what happens.

Â Okay?

Â Less variance, in our sample mean.

Â So these densities are getting tighter and tighter.

Â 12:06

Very low and a four, again the black is moved,

Â because it, it has to force this area to be 5% and

Â now our power is quite low.

Â 12:20

So I would highly recommend that you go through the code for this manipulate

Â experiment to understand how power works in this particular setting, it's, it's,

Â it's quite easy, but, basically, what you can see is power has a bunch of knobs.

Â Oops. And as you turn them,

Â the power changes in different ways,

Â and in the next slide, we'll summarize the various aspects of power.

Â But using the manipulate function like this,

Â you can actually experiment with it your self.

Â