Explore
For Enterprise
Join for Free
Log In

Arts and Humanities
Business
Computer Science
Data Science
Information Technology
Health
Math and Logic
Personal Development
Physical Science and Engineering
Social Sciences
Language Learning
Degrees
Certificates

Explore all of Coursera

Browse
Search
For Enterprise
Log In
Join for Free

4.3_A Review of Validity

Loading...

Positive Psychology: Character, Grit and Research Methods

University of Pennsylvania

4.3 (582 ratings) | 10K Students Enrolled

Course 3 of 5 in the Foundations of Positive Psychology Specialization

Learners discover how apply to research methods to their study of Positive Psychology. In this course, we study with Dr. Angela Duckworth and Dr. Claire Robertson-Kraft. Through an exploration their work "True Grit" and interviews with researchers and practitioners, you develop a research hypothesis and learn how to understand the difference between internal and external validity. You also begin to understand and apply the strengths and weaknesses associated with different types of measurements and evaluation designs. You then interpret the results in an empirical study. Suggested prerequisites: Positive Psychology: Martin E. P. Seligman’s Visionary Science and Positive Psychology: Applications and Interventions.

Reviews

4.3 (582 ratings)

5 stars
359 ratings
4 stars
124 ratings
3 stars
52 ratings
2 stars
30 ratings
1 star
17 ratings

AP

Jun 29, 2017

This is an excellent course as an introduction to reading and understanding research. As an early stages post grad student, I have benefitted enormously from this course.

NM

Feb 05, 2019

An exciting introductory course for understanding research design and analysis. Highly recommended for beginners in research. Thanks Dr. Duckworth and Dr. Kraft.

From the lesson

Using Research to Inform Your Work and the World Around You

4.3_A Review of Validity8:26

Angela Duckworth: 4.3 Grit & Parenting3:33

Research: Angela Duckworth: Consequential Validity4:35

Practitioner Interview: (4.3) Andrew Soren2:41

Taught By

Claire Robertson-Kraft, Ph.D.
Director, ImpactED
Angela Duckworth, Ph.D.
Christopher H. Browne Distinguished Professor of Psychology

Transcript

So we're going to go back to where we started that first week and

discuss how you can be a critical consumer of the research and

the media's interpretation of that research.

In your reading this week, I asked you to review a research article,

Eavesdropping on Happiness, and then a popular press article reviewing it.

We'll use this example to talk about how to be a critical consumer of the research

and the media.

Let's talk a bit about the study first.

The authors in this Eavesdropping on Happiness study

were interested in whether conversation type was related to well-being.

They observed 79 students naturally without direct interference.

They did this by using a digital audio recorder which allowed for

unobtrusive tracking of 300 waking recordings per student.

And they did this by 30 second recordings every 12.5 minutes over four days.

Then they coded conversations and

differentiated between different types of conversation.

So they looked at alone time, talking to others,

small talk and substantive conversations.

Then they assessed students' well-being using self reports, informant reports and

the satisfaction with life scale.

Students also took the big five inventory twice

to capture information on personality.

Scores were averaged with informants ratings of students on the same inventory.

The results of this study demonstrated that higher well-being was associated with

more conversations and less alone time.

Higher well-being was also associated with more substantive conversations and

less small talk.

These results remained consistent even when controlling for

personality type and whether data was collected on weekdays or weekends.

So the happiest participants were alone 25% less of the time.

And spent about 75% more time talking.

So how should we go about assessing this research study, and

the media's summary of it?

Well, remember the concept of validity, which means truth?

As John Keats puts it, beauty is truth, truth beauty.

That is all you know on earth and all you need to know.

We've talked about three types of validity which are critical to evaluating research.

Internal validity, external validity, measurement validity.

And today we'll introduce a fourth type, consequential validity.

First, internal validity.

Internal validity, as you'll recall, captures whether the study

can demonstrate that the independent variable causes the dependent variable.

And there are two main questions to consider when looking at

internal validity.

First, the direction of the effect.

So can you demonstrate that the independent variable

leads to the dependent variable, or could it be the other way around?

And second, could there be other variables that could explain the outcome that

you didn't actually measure?

So in the case of the Eavesdropping on Happiness article, did having more

conversations and less alone time actually lead to higher well-being?

Or was it the other way around?

Did having higher well-being actually lead you to

have more substantive conversations?

Or was there another variable, altogether,

that the authors didn't measure that led to well-being.

Now, the authors did control for some other personality variables captured

through the big five but, certainly not all of them.

So there could have been another variable.

I'm sure you all could think of many that could lead to increased well-being

that this study didn't assess.

And the authors can't account for other factors or

the direction of the effect because there wasn't random assignment to groups.

So this means we should be interpreting these results as correlational not causal.

So how well did the media article do?

Well, in the opening lines of this popular press article the author states that,

new research suggests that less small talk and

more substantive conversation causes increased happiness.

Later he does go on to state that correlation doesn't necessarily prove

causation.

Which was an important observation, but he does use misleading language throughout.

So he uses language like conversation may galvanize a previously unexamined life.

Implying causation.

Now this is one of the most important things to look for

in a popular press article.

As the media will often overstate findings as causal

when the study was only correlational.

Why you might ask?

Well because it certainly makes for a flashier article and better headlines.

The second type of validity to consider is external validity.

External validity captures whether the results will translate to other

populations.

So, for example, we need to consider if and

how results will translate to different settings, people, places, and times.

The Eavesdropping on Happiness article only looked at undergraduates,

which means there are limitations in external validity.

So the question remains, could this even be applied to non-undergraduates?

Or is there something about the types of conversations you have in college

that are associated with your well-being that wouldn't apply outside of college?

Now this is something that the media article doesn't address at all and

is oftentimes another one of the limitations that you'll find in these

types of media articles.

The media like to make it seem like the findings of an article are universal and

would apply in any setting, when often they are quite context specific.

The third type of validity we've discussed is measurement validity.

How well does the study capture its intended constructs?

The study uses validated measures with high reliability like the satisfaction of

life scale.

The big five inventory and

measures of well-being, which is the positive aspect of the study.

They also use multiple measures which increases the convergent validity.

Remember, no one measure is perfect, but using multiple measures is a solution to

get at that imperfection and reduce the error of any one measure alone.

However, the measures are still self report and informant measures,

which as we talked about in week two, have several limitations.

Perhaps there was social desirability bias and

students answered questions in a certain way.

Or perhaps there was reference bias in this group of undergraduates

meaning that they had a different reference point

about what well-being meant that wouldn't apply in other settings.

Now these limitations weren't touched on at all in the popular media article.

So as a critical consumer of the research it's something you need to look for.

And the final type of validity is consequential validity,

which refers to the positive or negative consequences of a measure or study.

We need to consider how measures should and

shouldn't be used based on their limitations.

So while this study was interesting and

useful, it's not clear how much we should make of the results.

We can't demonstrate the direction of the effect, the study was only conducted with

undergraduates, and there are limitations in self report and informant measures.

So we shouldn't go as far as to the popular media article suggestion and

conclude that real conversation

might be the necessary stimulant to galvanize a previously unexamined life.

It might be, but we'd need more research to demonstrate this was the case.

So this bring us to the end of the final video

introducing key concepts for this week.

There are a few other components for this week.

Again, researcher videos featuring Angela Duckworth applying these concepts to

her work.

Practitioner videos featuring practitioners applying these concepts

to their work.

A short quiz on key concepts, and two optional extension activities.

A peer review activity where you discuss what kind of research design

you would use to test your research question and hypothesis.

And explore the strengths and weaknesses associated with that design.

And an advanced activity where you read the popular media article on true grit and

discuss how effectively you think the author captured the key findings from

the study.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play

Coursera
About
Leadership
Careers
Catalog
Certificates
MasterTrack™ Certificates
Degrees
For Enterprise
For Government

Community
Learners
Partners
Developers
Beta Testers
Translators

Connect
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Tech Blog

More
Terms
Privacy
Help
Accessibility
Press
Contact
Directory
Affiliates