We are always using experiments to improve our lives, our community, and our work. Are you doing it efficiently? Or are you (incorrectly) changing one thing at a time and hoping for the best? In this course, you will learn how to plan efficient experiments - testing with many variables. Our goal is to find the best results using only a few experiments. A key part of the course is how to optimize a system. We use simple tools: starting with fast hand calculations, then we show how to use FREE software. The course comes with slides, transcripts of all lectures, subtitles (English, Spanish and Portuguese; some Chinese and French), videos, audio files, source code, and a free textbook. You get to keep all of it, all freely downloadable. This course is for anyone working in a company, or wanting to make changes to their life, their community, their neighbourhood. You don't need to be a statistician or scientist! There's something for everyone in here. ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ What have prior students said about this course? "This definitely is one of the most fruitful courses I have participated at Coursera, considering the takeaways and implementations! And so far I finished 12 [courses]." "Excelente curso, flexible y con suficiente material didáctico fácilmente digerible y cómodo. No importa si se tiene pocas bases matemáticas o estadísticas, el curso proporciona casi toda explicación necesaria para un entendimiento alto." "I wish I had enrolled in your course years ago -- it would have saved us a lot of time in optimizing experimental conditions." Jason Eriksen, 3 Jan 2017 "Interesting and developing both analytical and creative thinking. The lecturer took care to bring lots of real live examples which are fun to analyze." 20 February 2016. "... love your style of presentation, and the examples you took from everyday life to explain things. It is very difficult to make such a mathematical course accessible and comprehensible to this wide a variety of people!" ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Enrichment: An interview with Dr. Joe Kim (McMaster University)
Loading...
From the course by McMaster University
Experimentation for Improvement
374 ratings
From the lesson
Response surface methods (RSM) to optimize any system
This is the goal we've been working towards: how to optimize any system. We start gently. We optimize a system with 1 factor and we also show why optimizing one factor at a time is misleading. We spend several videos to show how to optimize a system with 2 variables.
Meet the Instructors
Kevin DunnFormerly: Assistant Professor
Chemical Engineering
So, this morning I'm here with Dr. Joe Kim.
Joe is a professor of psychology here at McMaster University, and
we're going to talk about his research area and
experimentation that he does in his research.
So, Joe, you, I've interacted with you for a few months at Mac, and
I know that you do a ton of work in experimentation,
in your research can you tell us a bit about that?
>> So, I direct the Applied Cognition Education Lab, and we're interested
in trying to understand how cognitive principles such as learning, memory,
attention can help us make informed decisions about evidence-based education.
So what I mean by that is, what is the best way to teach a particular concept to
a person and help them to really learn things for long term comprehension?
>> So if you say best way, how, that to me would be a tough one to evaluate.
How do you measure that as an outcome?
>> So, typically our outcome measure is performance on a comprehension test.
So a learner has to learn a set of information.
And depending on the experiment, that might vary.
And then we're typically interested in,
what is the immediate impact on their comprehension.
So how do they perform on the test.
Right away.
>> Right away. >> Okay.
>> But we're also interested in
the long term, and typically what we mean by the long term is
one week.
>> Okay. >> So
one week later we'll test, give them a different test, and see how much do
they actually maintain of that new learned information over that one week.
Ideally, we'd like to do that over even longer periods.
>> Right. >> But in running experiment,
it's typically difficult to get people to come back after months or, or even a year.
>> Fair enough.
Now, given that,
what are the sort of things you're changing up in your experiments?
>> Well we have several different lines of research.
One line is on multimedia learning.
And what I mean by that is, what is the best way,
for example, to design PowerPoint slides that will lead
to the most effective learning as measured by comprehension?
>> Sure.
>> So, the tip, in one type of experiment, for example, we might compare,
how do learners in three different groups learn the same information?
So, all three groups might listen to the same audio track of narration.
So that they're getting the same lecture.
>> So that factor is constant.
>> That factor is constant.
>> That factor is constant. In one group, they might be looking at
slides that are, are very typical, where they're very text heavy.
So, lots of bullet points of text.
And that's something that many people are really used to.
Another condition might have very minimal text
strong visuals that illustrate the main point that's being made on the slide.
>> Mm-hm.
>> And the third condition as a control might be,
nothing, they're just staring at a blank screen.
>> Okay. >> And then all three groups will be
tested for their comprehension on an immediate test and then one week later.
>> Right. Okay so, type of slides is one.
Any other factors that you would be investigating in those?
>> In those types of experiments?
>> Yeah, or in other research?
>> okay. So, in other research lines one of
the things that we're looking at is called transfer of knowledge.
So typically, when students are learning one
type of concept or information they might try to memorize that particular concept.
Then on a test, they're given an application.
>> Okay.
>> And that sort of stretches them.
So it's not something you can just memorize.
>> Right. >> And
one of the things that we're looking at is,
how do we help people get from memorizing a fact to actually applying it to
an application question that they've never seen before.
>> Right. >> So this transition is called transfer.
>> Right? And that's why we're looking.
>> Yeah. That's right.
And there are many different ways that we're looking at to try to
help this transfer process.
So for examples students, when they're learning statistics [COUGH]
one of the challenges is that the first week, you might be learning about t-tests.
Right.
>> And then all your practice problems, not surprisingly are about t-tests.
>> Sure. >> The answer is t-test and
students learn to memorize this formula and then they just plug in the numbers and
they get the answer right and then they feel like, I really understand this.
>> Right. >> Then the next week they
might learn about a Wilcoxon signed-rank test and then not surprisingly all
the questions that week, they're all about Wilcoxon signed-rank test.
>> Sure. >> And that's the answer.
And they learn that formula and they learn to plug it in.
>> Right. >> Now the challenge is on the exam,
I give you a question and the biggest, biggest part of the problem is,
well what sort of statistical test should I do on this?
>> Yeah.
>> And students have memorized the formula, they know how to,
kind of, if they plugged things in, but they don't really know how to
make the transfer process through this novel situation.
So, on the front end of teaching there's a few different manipulations we looked at.
So one thing that we have found that can help is for
example, instead of having a blog learning design where every answer during
your practice is going to be the t-test.
Or the Wilcoxon signed-rank.
In a mixed design, some of the practice questions you have,
the answer really is the t-test, but some of them are the Wilcoxon signed-rank, so
things that you learned in previous weeks are woven into the practice questions, And
students were exposed to mixed questions.
It's suppose to block questions even though you do
the exact same number of questions.
>> Yeah. >> In the end,
they perform much better in the final exam.
>> All right. Okay, that's so interesting.
Now, there's, there's another piece of work that you spoken with me
about before on the testing effect and the spacing effect-
>> Mm-hm.
>> In which I found more fascinating, can you tell us a more about that?
>> Yes so the testing effect suggests that if you're trying to
retain information for the long term-
>> Hm-mm. >> Tes-,
being tested is superior to restudying.
>> Okay, so simple reading of the-
>> Yeah, so the way that we do that is imagine that there is
this story passage that all the student's have to read and they're told,
you're trying to remember this information.
You're going to be tested on this maybe a few weeks from now.
>> Okay.
>> So really try to learn this information as best as you can.
And in one group, immediately after they read it,
they're given an opportunity to re-study.
So go over, read it again.
And in the other, in the testing group.
Instead having a chance to re-read and re-study.
>> Okay. >> They are actually tested on, on,
on the passage.
>> Right.
>> And in the short term the group that re-studies,
they actually do a little bit better than the group that's tested.
>> Right.
>> But in the long term, the group that has opportunities to be
tested as opposed to re-studying will outperform the study group.
>> Okay, so something during that test is causing the knowledge to be
a little bit more solidified.
>> [CROSSTALK] Yeah, so
this is something that's really up for theoretical interpretation.
One of the ideas in cognition is that every time you
practice recalling a memory, it strengthens that memory.
>> Right. >> And
there's evidence at a neurophysiological level, the cognitive level,
that this is actually the case.
In fact the best time to try to recall information is
just when you're about to lose it forever.
And repeating this process actually really helps you to consolidate.
And testing seems to tap into this process.
>> Right.
>> And it's important to note that it's not that final test is
the exact same test that you've been tested with all along.
>> Right. >> It's a completely different test.
>> Right.
>> So the take home message that I always tell students is that,
to incorporate the testing effect into your own study instead of
mindlessly reading and rereading and highlighting your notes over and over, and
putting in the hours and feeling like you've done a lot of work.
Actually do work by testing yourself, and
you're even better off by creating your own test questions.
>> So we're in a part of McMaster's Campus called the LIVE Lab which was just built
in 2014.
There's so much going on here.
Can you tell us a bit about this building?
>> Yeah, I'm really actually personally excited about working in the LIVE Lab.
So LIVE Lab stands for Large Interactive Virtual Environment.
>> Okay. >> And it's a 100 seat
performance theater with virtual acoustics.
So one of the exciting things is that you can make this
performance theater sound like the smallest room or
you can make it sound like a large concert hall or a large, cavernous church.
Anything that you want, depending on the needs of your experiment.
What's even more exciting is that 30 of these seats are wired so
that from the audience members you can directly measure things like EEG.
You can measure heart rate, galvanic skin response and
in the future we're also going to set up eye tracking and motion capture.
So, these are amazing sets of tools that you can use, and so
for my own research, I'm interested in teaching and learning.
So I look at this performance theatre as this is actually the world's most
advanced lecture hall.
>> Yeah.
>> And imagine different types of manipulations such as, imagine different
groups of people are exposed to different PowerPoint slides with the same narration.
We typically ask in addition to comprehension tests, we typically ask
people, you know, to, to rate their own learning performance or how well.
>> [CROSSTALK] On a scale from one to five. >> Yeah,
how well do you think these slides are helping you with your learning, or
you understanding, or you interest and engagement with the material?
And we have to ask them questions on the Likert scale, say one to five, and
they just simply put their subjective ratings, and
we've done that because it's the only tool we have available, but with the LIVE Lab
we can move beyond these subjective scales and actually look at physiological data.
So, we can look at things like galvanic skin response and
heart rate variability as measures of arousal.
We could look at EEG measures as measures of attention.
>> Right. >> So, I'm very excited about these new
tools that we look at for new dependent variable measures.
>> Wow. Those are going to be
some good case studies coming out of this.
>> I'm very excited.
Yeah, great.
Thanks, Joe. Appreciate the time.
>> Anytime.
>> Good luck with that research.
>> All right. >> Thanks.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
