Learners discover how apply to research methods to their study of Positive Psychology. In this course, we study with Dr. Angela Duckworth and Dr. Claire Robertson-Kraft. Through an exploration their work "True Grit" and interviews with researchers and practitioners, you develop a research hypothesis and learn how to understand the difference between internal and external validity. You also begin to understand and apply the strengths and weaknesses associated with different types of measurements and evaluation designs. You then interpret the results in an empirical study. Suggested prerequisites: Positive Psychology: Martin E. P. Seligman’s Visionary Science and Positive Psychology: Applications and Interventions.
4.3_A Review of Validity
Loading...
From the course by University of Pennsylvania
Positive Psychology: Character, Grit and Research Methods
386 ratings
University of Pennsylvania
386 ratings
Course 3 of 5 in the Specialization Foundations of Positive Psychology
From the lesson
Using Research to Inform Your Work and the World Around You
Meet the Instructors
Claire Robertson-Kraft, Ph.D.Director, ImpactED
Positive Psychology Center and Fels Insititute of Government
Angela Duckworth, Ph.D.Christopher H. Browne Distinguished Professor of Psychology
Positive Psychology Center
So we're going to go back to where we started that first week and
discuss how you can be a critical consumer of the research and
the media's interpretation of that research.
In your reading this week, I asked you to review a research article,
Eavesdropping on Happiness, and then a popular press article reviewing it.
We'll use this example to talk about how to be a critical consumer of the research
and the media.
Let's talk a bit about the study first.
The authors in this Eavesdropping on Happiness study
were interested in whether conversation type was related to well-being.
They observed 79 students naturally without direct interference.
They did this by using a digital audio recorder which allowed for
unobtrusive tracking of 300 waking recordings per student.
And they did this by 30 second recordings every 12.5 minutes over four days.
Then they coded conversations and
differentiated between different types of conversation.
So they looked at alone time, talking to others,
small talk and substantive conversations.
Then they assessed students' well-being using self reports, informant reports and
the satisfaction with life scale.
Students also took the big five inventory twice
to capture information on personality.
Scores were averaged with informants ratings of students on the same inventory.
The results of this study demonstrated that higher well-being was associated with
more conversations and less alone time.
Higher well-being was also associated with more substantive conversations and
less small talk.
These results remained consistent even when controlling for
personality type and whether data was collected on weekdays or weekends.
So the happiest participants were alone 25% less of the time.
And spent about 75% more time talking.
So how should we go about assessing this research study, and
the media's summary of it?
Well, remember the concept of validity, which means truth?
As John Keats puts it, beauty is truth, truth beauty.
That is all you know on earth and all you need to know.
We've talked about three types of validity which are critical to evaluating research.
Internal validity, external validity, measurement validity.
And today we'll introduce a fourth type, consequential validity.
First, internal validity.
Internal validity, as you'll recall, captures whether the study
can demonstrate that the independent variable causes the dependent variable.
And there are two main questions to consider when looking at
internal validity.
First, the direction of the effect.
So can you demonstrate that the independent variable
leads to the dependent variable, or could it be the other way around?
And second, could there be other variables that could explain the outcome that
you didn't actually measure?
So in the case of the Eavesdropping on Happiness article, did having more
conversations and less alone time actually lead to higher well-being?
Or was it the other way around?
Did having higher well-being actually lead you to
have more substantive conversations?
Or was there another variable, altogether,
that the authors didn't measure that led to well-being.
Now, the authors did control for some other personality variables captured
through the big five but, certainly not all of them.
So there could have been another variable.
I'm sure you all could think of many that could lead to increased well-being
that this study didn't assess.
And the authors can't account for other factors or
the direction of the effect because there wasn't random assignment to groups.
So this means we should be interpreting these results as correlational not causal.
So how well did the media article do?
Well, in the opening lines of this popular press article the author states that,
new research suggests that less small talk and
more substantive conversation causes increased happiness.
Later he does go on to state that correlation doesn't necessarily prove
causation.
Which was an important observation, but he does use misleading language throughout.
So he uses language like conversation may galvanize a previously unexamined life.
Implying causation.
Now this is one of the most important things to look for
in a popular press article.
As the media will often overstate findings as causal
when the study was only correlational.
Why you might ask?
Well because it certainly makes for a flashier article and better headlines.
The second type of validity to consider is external validity.
External validity captures whether the results will translate to other
populations.
So, for example, we need to consider if and
how results will translate to different settings, people, places, and times.
The Eavesdropping on Happiness article only looked at undergraduates,
which means there are limitations in external validity.
So the question remains, could this even be applied to non-undergraduates?
Or is there something about the types of conversations you have in college
that are associated with your well-being that wouldn't apply outside of college?
Now this is something that the media article doesn't address at all and
is oftentimes another one of the limitations that you'll find in these
types of media articles.
The media like to make it seem like the findings of an article are universal and
would apply in any setting, when often they are quite context specific.
The third type of validity we've discussed is measurement validity.
How well does the study capture its intended constructs?
The study uses validated measures with high reliability like the satisfaction of
life scale.
The big five inventory and
measures of well-being, which is the positive aspect of the study.
They also use multiple measures which increases the convergent validity.
Remember, no one measure is perfect, but using multiple measures is a solution to
get at that imperfection and reduce the error of any one measure alone.
However, the measures are still self report and informant measures,
which as we talked about in week two, have several limitations.
Perhaps there was social desirability bias and
students answered questions in a certain way.
Or perhaps there was reference bias in this group of undergraduates
meaning that they had a different reference point
about what well-being meant that wouldn't apply in other settings.
Now these limitations weren't touched on at all in the popular media article.
So as a critical consumer of the research it's something you need to look for.
And the final type of validity is consequential validity,
which refers to the positive or negative consequences of a measure or study.
We need to consider how measures should and
shouldn't be used based on their limitations.
So while this study was interesting and
useful, it's not clear how much we should make of the results.
We can't demonstrate the direction of the effect, the study was only conducted with
undergraduates, and there are limitations in self report and informant measures.
So we shouldn't go as far as to the popular media article suggestion and
conclude that real conversation
might be the necessary stimulant to galvanize a previously unexamined life.
It might be, but we'd need more research to demonstrate this was the case.
So this bring us to the end of the final video
introducing key concepts for this week.
There are a few other components for this week.
Again, researcher videos featuring Angela Duckworth applying these concepts to
her work.
Practitioner videos featuring practitioners applying these concepts
to their work.
A short quiz on key concepts, and two optional extension activities.
A peer review activity where you discuss what kind of research design
you would use to test your research question and hypothesis.
And explore the strengths and weaknesses associated with that design.
And an advanced activity where you read the popular media article on true grit and
discuss how effectively you think the author captured the key findings from
the study.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
