And these are illustrations of some of the more extreme examples of what I would call
the replicability crisis.
This is one of three crises in neuroscience that we're going to talk
about today, with a view towards how to avoid them, and
how to educate ourselves to make better and smarter inferences about brain images.
The replicability crisis is fueled by findings that are not true and
effects that might be true, but are not meaningfully large or
not reproducible from laboratory to laboratory.
The interpretability crisis concerns findings that might be true,
but they're not meaningful in terms of underlying neuroscience and
what we know about brains.
And finally, the translation crisis refers to this idea that we've done
many many studies, but it's very difficult to take brain imaging research,
like many kinds of research, genetics research and other research,
and bring that from science into the practical domain,
where we actually have commercial and clinical applications.
So this [LAUGH], yeah.
But what we'll see next is a funny example of one of
the studies recently that's really fueled this debate.
And it's a study of extra sensory perception published in one of the most
prestigious journals in psychology, General Personality and Social Psychology.
>> According to a study by Cornell Psychology Professor,
Daryl Bem, soon to be published in the Journal of Personality and
Social Psychology, there is quote, strong evidence for
extrasensory perception, the ability to sense future events.
I know you're thinking, Stephen, that's bulls [BLEEP].
>> [LAUGH] >> But on the other hand,
I know you're thinking, Stephen, that's bulls [BLEEP].
>> So many of these surprising claims are not true.
And how do we know that they're not all true?
Any one of them could actually be a surprising but true finding.
And one way that we can know is to look at studies across different fields,
across the sciences.
So here you see plot from studies from the space sciences,
and geo sciences through to social sciences, immunology,
climatology, psychiatry, psychology down at the bottom.
And what we're looking at,
is the proportion of papers that claim to support the hypothesis they tested.
And these claims, well, so what's the base rate
of having a hypothesis that you end up supporting?
It's probably low, but here most studies, and in fact in psychology and
psychiatry, nearly all studies end up supporting the claims that they tested.
And one explanation is that they're choosing the claims to support,
to fit the data.