This course covers the design, acquisition, and analysis of Functional Magnetic Resonance Imaging (fMRI) data. A book related to the class can be found here: https://leanpub.com/principlesoffmri

Loading...

From the course by Johns Hopkins University

Principles of fMRI 1

329 ratings

This course covers the design, acquisition, and analysis of Functional Magnetic Resonance Imaging (fMRI) data. A book related to the class can be found here: https://leanpub.com/principlesoffmri

From the lesson

Week 4

The description goes here

- Martin Lindquist, PhD, MScProfessor, Biostatistics

Bloomberg School of Public Health | Johns Hopkins University - Tor WagerPhD

Department of Psychology and Neuroscience, The Institute of Cognitive Science | University of Colorado at Boulder

Hi. In this module,

Â we're going to continue talking about the multiple comparison problem in FMRI.

Â In particular, we're going to focus on methods that correct for

Â the family-wise error rate.

Â So the family-wise error rate is the probability of making one or

Â more Type I errors in a family of tests, under the null hypothesis.

Â And a Type I error is when we reject a null hypothesis and

Â we really shouldn't have done that.

Â So there are a number of family-wise error rate controlling methods

Â that are used in neural imaging.

Â They include the classic Bonferroni correction, Random Field Theory,

Â and permutation tests.

Â In this module, we'll talk a little bit about Bonferroni correction and

Â Random Field Theory.

Â Now let's let Hnot of i be the hypothesis that there's no activation in voxel i

Â where I take values from 1 to m, where m is the total number of voxels.

Â So basically Hnot i is just the voxel wise null hypothesis of no activation.

Â And now let's let Ti be the test statistic at voxel i.

Â So we conducted a test of the null hypothesis at each voxel, so

Â we have Ti And those are the values that make up our statistical map.

Â Now the family-wise null hypothesis, which I'm just going to call Hnot here,

Â state that there's no activation in any of the m voxels.

Â So basically we're just assume that there's no activation anywhere

Â across the brain.

Â If this is true, then Hnot is true.

Â So, for Hnot to be true, there has to be activation in none of the areas.

Â So, mathematically, we can write this as the intersection of the Hnot of i's.

Â So, the H knot has to be true in each i, in each voxel.

Â So, if we reject the single voxel null hypothesis,

Â we're going to reject the family-wise null hypothesis.

Â So because basically if all the, if any of the null hypothesis are,

Â the individual null hypothesis are rejected,

Â then the family-wise null hypothesis is also rejected.

Â So a false positive at any voxel will give a family-wise error.

Â So let's assume that the family-wise null hypothesis is true.

Â Then we want to control the probability of falsely rejecting Hnot at

Â some level alpha.

Â So basically what we want to do here is we want to control

Â the probability that any of the test statistics at

Â any of the voxels over the entire brain is above some value u.

Â And so we want, because if it's above u we're going to reject that

Â the null hypothesis at that voxel, and we don't want to do that,

Â because then we're going to get a false positive.

Â So basically we want to make the probability that Ti is bigger than u,

Â make that controlled by some value alpha.

Â Say 0.05.

Â So what we need to find is find the value of u which controls

Â the family-wise error rate at this particular level.

Â So the Bonferroni correction is the classic way of doing that.

Â And so in Bonferroni Correction we choose a threshold u, so

Â that the probability that any of these test statistics is above u

Â is less than alpha over m where m is the total number of voxels.

Â So if this is true,

Â then this controls the family-wise error rate as well because the family-wise error

Â rate it the probability that any of the test statistics is above u.

Â Which according to Boore's inequality it's the sum that any of them are above u,

Â which according to a threshold we choose is controlled by alpha.

Â So, the Bonferroni correction this simple math shows that it controls

Â the family-wise error rate at alpha.

Â So for example if we have ten tests and

Â we want to control the family-wise error rate at 0.05, then we should control each,

Â we should choose u, so that each test is going to control that 0.005 and

Â now if we have 100,000 tests we needed to divide by 100,000 so that

Â the threshold will become increasingly stringent as we do more and more tests.

Â So here's an example, let's say that we generate iid normal

Â (0,1) data, so over a 100x100 grid.

Â So in this case we have 10,000 pseudo voxels here,

Â each that follow a standard normal distribution.

Â And so here's a picture of that.

Â Now if we threshold this at u=1.645,,

Â this would be the 95th percentile of the standard normal distribution.

Â In this case we would get 500 false positives because this

Â is 0.05 times the total number of voxels, which is 10,000.

Â So we're going to get the salt-and-pepper pattern where white indicates that

Â something was above the threshold, and black indicates that it was below.

Â In this case we're not really controlling very well for false positives, so

Â what we need to do is a more stringent way.

Â So, we have approximately 500 false positives here.

Â So, to control the family-wise error rate at 0.05 the Bonferroni

Â correction would have to be at 0.05/10,000.

Â So we have to control for the fact that we're doing 10,000 tests.

Â And so if we do that.

Â Now the threshold instead of being 1.645 is now equal to 4.42.

Â So it's a much more stringent amount of evidence that needs for

Â us to reject a null hypothesis.

Â And if we do this we get no false positives at all.

Â So indeed, if we were to repeat this sort of simulation 100 times, on average only

Â 5 out of every 100 generated data sets would have one or more values above u.

Â And so basically, the probability of us getting any false positives among

Â the 10,000 tests is only 5%, so we'd have to do this whole exercise 100 times.

Â And only one set of every 20 would we get one or more false positives.

Â So this is a very, very stringent control over the false positive rate.

Â And so of course this is really great if you're worried

Â about getting false positives.

Â However, if you have true activations in this grid,

Â it's going to be very hard to detect them.

Â So there's sort of a tradeoff here between the ability to detect activations and

Â control the family-wise error rates.

Â This is going to be a very stringent way.

Â So, we're going to wind up losing a lot of activations if we use

Â the Bonferroni correction.

Â So, the Bonferroni correction, as I just mentioned, is very conservative,

Â it results in very strict significance levels.

Â So, this leads to a decrease in the power of the test.

Â And this the probability of correctly rejecting a false null hypothesis and

Â greatly increases the chance of getting false negatives.

Â And so in general, it's also not optimal for correlated data, and

Â most fMRI data has significant spatial correlation.

Â So the number of independent tests are actually much fewer than the number

Â of voxels.

Â So we may be able to choose a more appropriate threshold by using information

Â about the spatial correlation present in the data.

Â One way of doing this is to use random field theory.

Â Random field theory allows one to incorporate the correlation

Â in spatial correlation into the calculation of the appropriate threshold.

Â And it's based on approximating the distribution of the maximum statistic

Â over the entire image.

Â So what does this mean?

Â Well what's the link between the family-wise error rate and

Â the maximum statistic?

Â Well the family-wise error rate is the probability of getting

Â a family-wise error.

Â So this is the probability that any of these T values,

Â T statistics, exceeds u under the null hypothesis.

Â So now I want to claim that this is equal to the max, the probability that

Â the maximum t statistic is above u, because if the maximum is above u,

Â then there is a t statistic that is above the threshold.

Â If the maximum statistic is below u,

Â then there is by default no tests statistics that are above u.

Â Because the maximum is the biggest value.

Â So if we're interested in the probability of any t statistic exceeding u,

Â it's enough for

Â us to look at the probability of the max t statistic exceeds u under the null.

Â So if you want to control the family wise error rate we simply need to find

Â the distribution for the max t statistic and threshold using that.

Â So we choose the threshold u,

Â such as the max only exceeds it alpha percent of the time.

Â So how do we do that?

Â Well random field theory is one way of approximating

Â the tail of the max statistic.

Â And so a random field is a set of random variables defined at every point

Â in some D-dimensional space.

Â In our case it's usually a three-dimensional space of the brain.

Â And so we're mostly working with what is called Gaussian random fields.

Â And so Gaussian random field has a Gaussian distribution, or

Â a normal distribution, at every point and every collection of points.

Â And so a Gaussian random field is like any normal distribution defined by its mean

Â and covariance.

Â In this case, it's the mean function and the covariance function.

Â What we do in neuro imaging is we consider a statistical image,

Â the one with all the t statistics,

Â to be a lattice representation of the continuous random field.

Â And using random field methods we're able to approximate the upper tail

Â of the maximum distribution,

Â which is the part we need in order to find the appropriate threshold.

Â And also simultaneously we can account for

Â the spatial dependence inherent in the data.

Â And so that's a useful thing in the neuron imaging context.

Â Let's consider that we have some random field z(s) defined on some space, and

Â in our example let's just assume that it's a two dimensional space.

Â And so here we have the random field, and then on the left we see sort

Â of a heat map of the random field, and on the right, we see a mesh plot of it.

Â So basically, every spot in the two dimensional lattice,

Â we have some statistic value that follows a random field.

Â So when we work with random field theory,

Â we have to define something called the Euler Characteristic.

Â The Euler Characteristic is the property of a random field of an image

Â after it's been thresholded.

Â So basically what the Euler Characteristic does, in layman's terms,

Â is it counts the number of blobs.

Â The number of coherent areas minus the number of holes.

Â And at the high threshold it just counts the number of blobs.

Â So what does this mean?

Â Number of blobs, the number of holes?

Â Well let's look at the random field that we have here to the left and

Â let's say that we threshold it at the value u equal to .5.

Â That means that any value that's above .5 was set equal to one and

Â anything below .5 is set equal to zero.

Â Then we get the map on the right top here,

Â which is just a lot of white within the black there.

Â So here the Euler characteristic is going to be 27 because

Â 28 coherent islands of activation here, which I'm calling blobs.

Â And there's one hole, you see in the bottom.

Â There's a slight hole in one of the blobs.

Â And so it's going to be 27 different blobs minus holes,

Â so that's the Euler characteristic in that case.

Â If we go to the middle one here, we're thresholding at 2.75.

Â In this case, we only get two blobs and no holes, so the Euler characteristic is two.

Â Finally, if we go to u = 3.5, we get a single blob, and

Â the Euler characteristic is 1.

Â So the Euler characteristic is a property of this image after we've thresholded it.

Â So how do we use the Euler characters to control for the Family Wise Error Rate.

Â They seem to be far removed from each other.

Â Well it turns out that we've already determined that

Â there's a link between the family-wise error rate and the max T statistic.

Â So, the Family Wise Error Rate is equal to the probability

Â that the max T statistic is above U.

Â I claim that if the max statistics is above u, then we're going to one or

Â more blobs.

Â Because, if we're thresh holding at u, we're going to have one or

Â more areas that are white.

Â That are going to be deemed significant.

Â And so, in this case, basically if the max statistic's above u,

Â we're going to have one or more blobs.

Â And let's just assume for sake of argument that no holes exist.

Â In this case, we're actually interested in the probability that

Â the Euler Characteristic is bigger than or equal to one.

Â That means that we have one or more blobs.

Â If we assume that there's never more than one blob, then this probability is

Â approximately equal to the expected Euler Characteristic.

Â So now we have that the link between the family-wise error rate and

Â the Euler Characteristic.

Â So the family-wise error rate is actually just the expected Euler Characteristic.

Â Now, this seems to have complicated the problem a lot,

Â because how would we know what the expected Euler Characteristic is?

Â Well the good news is that, actually closed form results exist for

Â the expected Euler Characteristic for Z T, F and X squared continuous random fields.

Â So we can kind of stand on the shoulders of the people who have already derived

Â these results and use them to control the family-wise error rate.

Â So for three dimensional Gaussian Random Fields this is the result for

Â the expected Euler Characteristic.

Â It takes a same-what complicated formula, where R is V over

Â FWHM in each of x, y, and z direction.

Â So V is the volume of the search region, so the number of voxels basically that

Â we're searching over and the full width at half maximum represents the smoothness

Â of the image estimated for the data in each direction.

Â So R is sometimes in the nomenclature is called a resolution element, or resel.

Â So basically, using this result, we can find that for large u,

Â the family-wise error rate is roughly equal to this.

Â So we can choose a threshold u to control the family-wise error rate.

Â And so what are some properties of this equation?

Â Well, As u increases, as the threshold increases,

Â you can see that the family-wise error rate will decrease if u is large.

Â So, that's a good thing, because if we make the threshold more stringent,

Â the family-wise error rate should go down.

Â So this is a useful property.

Â Similarly as V increases so the number voxels that we're controlling for

Â increases, the family-wise error rate will also increase and this is again

Â a useful property because the more test that we're comparing simultaneously,

Â the more likely are we to make a family-wise error.

Â Finally as the smoothness increases, then the family-wise error rate will decrease.

Â And this is again a useful property because if we have a very smooth image,

Â we would expect adjacent voxels to behave similarly and

Â will have less independent tasks.

Â So even though this formula looks kind of ugly and hard to grasp,it has

Â a lot of properties that are useful for controlling the family-wise error rate.

Â So what are some assumptions that we need to hold in order to use this random

Â field theory?

Â Well, we have to assume that the entire image is either a multivariate Gaussian,

Â or derived from some multivariate Gaussian image.

Â So that includes chi-squared, T, and F-distributions, so those are the kind of

Â distributions that we're often interested Sit and working with.

Â The statistical image must also be sufficiently smooth to

Â approximate a continuous random field.

Â And so the family-wise error rate has to be at least twice the voxel size, and

Â so typically to analyze error rates, we want

Â the smoothest to be at least 3 to 4 voxel sizes, for this to work really well.

Â And also the amount of smoothness is assumed to be known, and

Â if the estimate can be biased when the images are not sufficiently smooth.

Â Also as we saw when deriving these results,

Â there are several layers of approximations that have to be made.

Â So that's the end of this module where we talked about two different methods for

Â controlling the family-wise error rate.

Â The first is the classic Bonferroni correction.

Â Which winds up being a little bit conservative because it doesn't take its

Â spacial relationships into consideration.

Â And then we also talked about random field theory, which is probably the most popular

Â way of controlling for family-wise error rate in neuro imaging.

Â And this controls a little bit for the spacial smoothness.

Â In the next module we'll talk about another type of way for controlling for

Â multiple comparison, which is the false discovery rate.

Â Okay, I'll see you then, bye.

Â Coursera provides universal access to the worldâ€™s best education,
partnering with top universities and organizations to offer courses online.