Learn fundamental concepts in data analysis and statistical inference, focusing on one and two independent samples.

Loading...

From the course by Johns Hopkins University

Mathematical Biostatistics Boot Camp 2

34 ratings

Johns Hopkins University

34 ratings

Learn fundamental concepts in data analysis and statistical inference, focusing on one and two independent samples.

From the lesson

Techniques

This module is a bit of a hodge podge of important techniques. It includes methods for discrete matched pairs data as well as some classical non-parametric methods.

- Brian Caffo, PhDProfessor, Biostatistics

Bloomberg School of Public Health

Hi, my name is Brian Caffo, and this is Mathematical Biostatistics Boot Camp, lecture 12. We're going to be talking about nonparametric statistics.

Okay, in this video, we're going to be talking about nonparametric tests, which include the sign test, which is useful for paired tests, which is kind of related to McNamar's test. Signed rank test which is also for paired data. we'll talk about Monte Carlo versions of that test. We'll talk about independent group tests, and then from that the Mann/Whitney test, and Monte Carlo versions of that, and then their relationship to permutation tests, which are very closely related.

Okay, so very briefly we're going to be talking about here about so-called non-parametric tests, but very kind of classical non-parametric tests, and these are often called distribution free. That of course doesn't mean that they're assumption free. They, they involve assumptions, for example, sampling assumptions such as [UNKNOWN].

but they require fewer assumptions than parametric methods. they have a tendency to focus a little bit more on testing rather than estimation. but, which maybe is a problem. But there are estimation techniques that follow. though they also tend not to be very sensitive to outlying objects. [UNKNOWN]

and they're especially useful for data like ranks, if the data actually come in the form of ranks. Because they often involve transforming data to ranks.

the, they're not uniformly wonderful, because they do throw out some information, which is their, their problem.

so then, and, and, because of that, they may wind up being less powerful than their parametric counterparts when the parametric assumptions are true of course.

and then, you know, for larger sample sizes, they become about as efficient as their parametric counterparts, so they, they are, are, are pretty, pretty good tests.

So here's some data from this wonderful book from Rice called, Mathematical Statistics and Data Analysis. I highly recommend this book. I use the second addition, I don't know if that's what they're still on. Anyway, this, this data concerned 25 fish. They were taking mercury levels from the fish in parts per million. And they had them at two locations on each fish, so, fish one, they had a measurement of 0.32 and 0.39, and so on. Okay, and then just for reference here, I've added the difference between those two measurements, subtracting the P from the SR, one each time. And then I'll show the ranking and the signed ranks, I'll explain those in a minute, but this is the data we're going to use as motivating data. Here we want to test whether the mercury levels taking, taken at location P differ from those of, of location SR where here

you know we're, we're, we're taking two measurements per fish. Trying to control for the fish to fish variability by taking both measurements on, on the same, on each fish. Having each fish serving, it's, it's own control. But we're going to be talking about non parametric tests, so what we're interested in or concerned about Is the validity of the assumptions that go into typical tests, such as normality.

Okay, so let's let Di be the difference for each fish. In this case, I'm subtracting P minus SR, and then let's let theta be the population median of the difference, of the, of the differences Di. And we want to test whether that median is zero versus the median being non-zero. Now, now by the definition of the median, if theta equals 0, only if the probability of a difference being greater than that value 0 is exactly 0.5, right? That's the definition of a median. Same thing for less than zero. Theorem.

and, so, so, as a test statistic why don't we just count the number of times that d is bigger than zero. And if it's excessively large or excessively small then that's going to to dispute the idea, that, that the median is exactly zero.

so just as an example, if all the differences were positive. Then zero couldn't be the median. because you wouldn't expect a large sample, where every single measurement was larger than the population median. Okay. Anyway then X here, each difference, if we're assuming the fish pairs are IID.

then each difference is in, is a, is a coin flip, with a 50% chance of being above the median. Or a 50% percent chance of being below the median. So the number of times it's larger than 0 is binomial. and in P, and in this case, P is 0.5. So our sign test just tests whether P is 0.5 using this data X, and you can do an exact binomial test, like we've talked about before.

Okay, so let's go back to our example. So theta is the median difference between, of P minus SR, our null hypothesis is that theta eqauls zero, versus the alternative that it's different from zero. The number of instances where the difference is bigger than zero, in this case go back to the table, or so that would be 15 out of 25 fish.

And the binomial test then is the question of whether 15 is large 15 positive instances out of 25 is, is large. Now, our expected number out of 25 is 12.5, so you know, 15, we don't know whether 15 is, is excessively large. Well, in this case, it turns out that, no, 15 is not excessively large. it has a, you know? A .42% chance of happening under the null hypothesis for a two sided test, in this case.

you know? And, again, you know? We could have used a large sample test. I don't know why, because we can do an exact test in this case. But we could have used a large sample test. And that's prob.test. And you get a chi-squared statistic then of 0.64, a p-value that's quite similar, 0.42.

a, a, any rate, the idea is that this is simply just testing are, if you're going to assume that levels of one are higher than the other, then you're just going to count the number of pairs from the matched pairs, where it's higher. And is that excessively large relative to a coin flip for each pair? So that's the sign test. And you might be wondering what's wrong with this, so let's discuss some, some potential problems with these tests.

Because it doesn't use very many assumptions, right? It doesn't, we aren't using very many assumptions. We're using that the Fisch pairs are IID,

but let's talk about what, what may be some of the problems. Okay, so the biggest problem is of course that the magnitude of differences is discarded, so it's potentially not as powerful as you would hope, right? You know, it'd be different if all, you know, maybe if you only got half of them as being positive, but all the ones that were Positive, or much larger differences, and all the ones that were negative were really small differences, then that would be different than if they were kind of spread equally between, above and below 0.

so I would say that. But then the other thing I would mention is there's nothing specific about 0. You could have tested any mean, theta equal to theta 0 By calculating the number of times the difference is bigger than any specific value, testing whether that the median is bigger than any specific value. What's interesting about that, then, you know, we won't talk about this, but it is kind of interesting, right, that you can do this for any value of theta. So that means you can find the values of theta for which you fail to reject, and the values of the theta for which you reject.

And then of course, if you can do that, you know, by a grid searcher, say, something like that, then you can invert that and get a a confidence interval for, for the median. And so this is kind of an interesting way, a very highly non-parametric way to get a confidence interval for the median of a set of data.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.