Explore
For Enterprise
Join for Free
Log In

Arts and Humanities
Business
Computer Science
Data Science
Information Technology
Health
Math and Logic
Personal Development
Physical Science and Engineering
Social Sciences
Language Learning
Degrees
Certificates

Explore all of Coursera

Browse
Search
For Enterprise
Log In
Join for Free

Module 1A: Crises in neuroscience and psychology: Problems and solutions

Loading...

Principles of fMRI 2

Johns Hopkins University

4.7 (114 ratings) | 7.7K Students Enrolled

This course covers the analysis of Functional Magnetic Resonance Imaging (fMRI) data. It is a continuation of the course “Principles of fMRI, Part 1”

Reviews

4.7 (114 ratings)

5 stars
86 ratings
4 stars
23 ratings
3 stars
4 ratings
2 stars
1 ratings

JM

May 24, 2019

Very good course. Best week is the last week. Its does a good overview of fMRI Statistical analysis and design of experiments.

PA

Sep 29, 2016

Thanks a lot, both curses were very interesting and will help a lot in future research!

From the lesson

Week 1

This week we will discuss psychological and behavioral inference, as well as advanced experimental design.

Module 1A: Crises in neuroscience and psychology: Problems and solutions14:00

Module 1B: Crises in neuroscience and psychology: Problems and solutions14:38

Module 2A: Pitfalls and Biases in Interpretation: Circularity, Reverse inference, Voodoo, and Beyond11:54

Module 2B: Pitfalls and Biases in Interpretation: Circularity, Reverse inference, Voodoo, and Beyond13:25

Module 3: Meta Analysis10:36

Module 4: Using meta-analysis to improve inference12:08

Module 5: Optimizing acquisition for your study goals18:27

Module 6: Resting-state fMRI7:46

Taught By

Martin Lindquist, PhD, MSc
Professor, Biostatistics
Tor Wager
PhD

Transcript

[SOUND] This module, we're going to

talk about the basics of what we can and

can't say with FMRI data.

And we're going to do this through the lens of crises in psychology and

neuroscience.

There are many surprising claims that have been made in the literature,

claims like this one, from the New York Times.

In this study, which is written up as a New York Times op-ed piece,

the authors claim to have scanned a number of brains of Democrats and

Republicans, and they claim to be able to make inferences about

what somebody's attitudes are, depending on what their brain scan show.

So for example, they write, the two areas in the brain associated with anxiety and

disgust, the amygdala and the insula, were especially active when men viewed

Republican, and based on that, they're inferring that the voters sense peril and

threat and disgust, and feel disgust.

And also emotions about Hilary Clinton are mixed,

based on activation in the amygdala and the nucleus accumbens,

which is supposed to be related to reward, and so on.

Here's another surprising claim.

In this article, Martin Lindstrom scanned people viewing their iPhones.

And what he writes is most striking of all was the flurry of activation

in the insular cortex of the brain, which is associated with feelings of love and

compassion.

It was discussed in the previous article.

Subjects brains responded to the sound of their phones as they would respond to

the presence or proximity of a girlfriend, boyfriend, or family member.

The subjects love their iPhones.

Many of these claims are demonstrably false, others are unlikely to be true, and

this isn't limited to brain imaging.

This is an endemic problem, in psychology, psychiatry, neuroscience, genetics,

biology and other fields.

And these are illustrations of some of the more extreme examples of what I would call

the replicability crisis.

This is one of three crises in neuroscience that we're going to talk

about today, with a view towards how to avoid them, and

how to educate ourselves to make better and smarter inferences about brain images.

The replicability crisis is fueled by findings that are not true and

effects that might be true, but are not meaningfully large or

not reproducible from laboratory to laboratory.

The interpretability crisis concerns findings that might be true,

but they're not meaningful in terms of underlying neuroscience and

what we know about brains.

And finally, the translation crisis refers to this idea that we've done

many many studies, but it's very difficult to take brain imaging research,

like many kinds of research, genetics research and other research,

and bring that from science into the practical domain,

where we actually have commercial and clinical applications.

So this [LAUGH], yeah.

But what we'll see next is a funny example of one of

the studies recently that's really fueled this debate.

And it's a study of extra sensory perception published in one of the most

prestigious journals in psychology, General Personality and Social Psychology.

>> According to a study by Cornell Psychology Professor,

Daryl Bem, soon to be published in the Journal of Personality and

Social Psychology, there is quote, strong evidence for

extrasensory perception, the ability to sense future events.

I know you're thinking, Stephen, that's bulls [BLEEP].

>> [LAUGH] >> But on the other hand,

I know you're thinking, Stephen, that's bulls [BLEEP].

>> So many of these surprising claims are not true.

And how do we know that they're not all true?

Any one of them could actually be a surprising but true finding.

And one way that we can know is to look at studies across different fields,

across the sciences.

So here you see plot from studies from the space sciences,

and geo sciences through to social sciences, immunology,

climatology, psychiatry, psychology down at the bottom.

And what we're looking at,

is the proportion of papers that claim to support the hypothesis they tested.

And these claims, well, so what's the base rate

of having a hypothesis that you end up supporting?

It's probably low, but here most studies, and in fact in psychology and

psychiatry, nearly all studies end up supporting the claims that they tested.

And one explanation is that they're choosing the claims to support,

to fit the data.

So this is related to what's called the studies publications bias or

the File Drawer Problem, which is a problem throughout sciences.

And the idea is that studies with negative results go in the file drawer,

you never see them, but studies that happen to find significant results,

even if they're not true or not replicable, end up getting published.

And because of this false findings are like rumors,

they're easy to spread, they're hard to dispel.

And one positive finding can cause a ripple that ends

up influencing the field for years to come.

And if those findings are false, this is obviously a dangerous practice.

One of the most famous examples of this is recent

controversy over whether autism is caused by vaccines.

And there was one publication that claimed to had find this.

It was picked up by the media and by several celebrities and

spread very widely.

And in spite of many, many large scale studies failing to find such associations,

it continues to be an idea that's prevalent among certain sectors

of the community, and there's a lot of harm that's caused by that.

The backlash against these replicability issues has been quite strong and

it's not always very specific.

So in this article, Ingfei Chen is saying that the vast majority of brain research

is drowning in uncertainty.

Not just a few isolated studies, but many, many areas of neuroscience,

and while there's some truth to this, there's some danger in this also.

Here are some of the important papers that look at sample size issues,

that look at issues of finding interactions and

complex effects where it might be difficult to demonstrate such effects, and

really critique then of all kinds of research by John Ioannidis.

What you see here, Why Most Published Research Findings are False.

So, these claims about replicability issues have been applied

really broadly across fields.

And this is one of the papers that's been influential in the brain imaging field.

It was originally called Voodoo Correlations in Social Neuroscience,

which was a targeted attack really on replicability and

social neuroscience studies that look at correlations between brain activity and

measure of personality, or attitudes, or behavior.

And they subserviently changed the title to Puzzlingly High Correlations in

Social Neuroscience, but it really caused a big stir.

And Sharon Begley was one of the early advocates of brain imaging research in

the news who covered a lot of these stories, and here she writes

the neuroscience blogosphere is crackling with- so far- glee over this paper.

And it really it made a big stir.

One question is whether this is caused, that these problems are caused,

by a few bad apples.

So one example recently is Diederik Stapel,

who had a series of very high profile findings in science and

other journals, and subsequently was found to have committed fraud and

admitted to fabricating data from a number of papers.

And this is a terrible thing, but

it's unfortunate that it happens once in a while, and we can ask, are all these

problems really just caused by a few scientists who are overtly unethical?

And I think this is not the case,

this isn't just caused by isolated breaches of scientific integrity.

It's caused by really a widespread bias even by well-meaning scientists.

So, we have to educate ourselves on what the sources of bias are, and

then how to work so that we minimize those sources of bias and can overcome them.

There are many reasons for this, but let's look at what Ioannidis says about

why most published research findings are false, and what are the factors involved.

So here's some risk factors for false positives.

One is smaller studies, and when we apply this to neuroimaging studies specifically,

neuroimaging studies are very expensive to collect and analyze,

and so they have been quite small by comparison to other studies.

The low prior probability of the effect being true, or at least very large,

the ESP example that we looked at from Bem's research is one such example.

Some people believe in ESP, other people don't, but there's no known mechanism and

even Bem says he doesn't know how it works.

And if there's no known physical mechanism,

then it's less likely that any phenomenon is true a priori, and

that makes findings that they're surprising in this way.

That find things with no mechanism less likely to be true.

So we need more evidence to demonstrate that they are, if indeed they are.

Three is the number of tests conducted.

We'll talk more about this in this lecture and the next lecture, but in neuroimaging,

we have many voxels that we test, sometimes hundreds of thousands of voxels.

We can test multiple contrasts, multiple maps and

so the risk of false positives is very great.

And in addition, when we pick out the winners,

the effect sizes of those voxels look much larger than they actually are.

The next factor is flexibility in design, analysis choices, and outcomes.

And in neuroimaging there are many analysis pathways and options.

And we'll look at some of the impact of that and false positive findings as well.

Next, is financial interest.

So, it's financial interest and also prestige interest.

We all have a stake in supporting the things that we want to be true.

And many of us have a stake, our status, and maybe our self-worth, is on the line

when we test our scientific findings and we want our favorite effect to pan out.

So we really have to guard against this.

It's costly to be wrong in many cases, even if we don't think we're biased and

wehave the best intentions, and we really have to keep that in mind.

And one of the interesting related findings related to this is that there

was a meta-analysis published that showed that significant predictor of how large

a drug effect is in a clinical trial, is whether a drug company funded that study.

And finally, the competition to publish.

In hot fields, there tend to be more false positives, and

possibly in more prestigious journals as well.

So don't we all want to publish in Nature and Science.

Well, to publish in Nature and Science,

I don't know how to publish in Nature and Science, but if I did,

I might say that you have to have something that's both very surprising.

You have to have a very surprising finding.

Let's look now at selection bias in more detail, and this is going to be the core

of a number of problems that we have to deal with in neuroimaging, and

they're resolvable.

And so first of all, studies and tests with positive findings are reported and

those without positive findings are ignored.

This is called the file drawer problem.

But there are multiple types of selection biases, and

they include the publication bias that I talked about a minute ago.

Also biases in experiments, inflexibility in which subjects to include,

how many subjects to run, when you stop running subjects, and all those

sources of flexibility, if you're choosing them after you observe the data,

are sources of false positives and increase inflated effect sizes.

Flexibility in the model, which outcome do you choose,

which covariance do you include, and which modeling procedures do you employ?

And finally, the problem of voxel selection bias.

So which voxels do I choose out of the many voxels that I tested to actually look

at and interpret.

So this is the interpretability crisis now, we'll talk a little bit about this.

And this relates to findings that might be true but are not meaningful.

Here are a couple of the classic papers that deal with inferences about cognitive

neuroscience, and what we can infer about function from looking at brain activity.

And the first one is a really old paper now by Martin Sarter and Berntson and

Cacioppo called Toward Strong Inference in Attributing Function to Structure, and

they discuss some of the difficulties in this inferential process.

Foreshadowing a lot of these debates later on.

And secondly, this is an article by Max Coltheart,

which spawned a series of responses, and what he asked is,

perhaps functional neuroimaging hasn't told us anything about the mind so far.

He challenged the field to come up with an example where brain imaging

supported a cognitive theory, or was critical evidence

in deciding whether a cognitive theory is true or false, or which theory is true.

And there were number of responses, and

there was a redux of this later by Mara Mather and colleagues.

And I invite you to go and look up some of those papers for many interesting answers,

and we'll talk about a couple examples later in the course.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play

Coursera
About
Leadership
Careers
Catalog
Certificates
MasterTrack™ Certificates
Degrees
For Enterprise
For Government

Community
Learners
Partners
Developers
Beta Testers
Translators

Connect
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Tech Blog

More
Terms
Privacy
Help
Accessibility
Press
Contact
Directory
Affiliates