So, this morning I'm here with Dr. Joe Kim. Joe is a professor of psychology here at McMaster University, and we're going to talk about his research area and experimentation that he does in his research. So, Joe, you, I've interacted with you for a few months at Mac, and I know that you do a ton of work in experimentation, in your research can you tell us a bit about that? >> So, I direct the Applied Cognition Education Lab, and we're interested in trying to understand how cognitive principles such as learning, memory, attention can help us make informed decisions about evidence-based education. So what I mean by that is, what is the best way to teach a particular concept to a person and help them to really learn things for long term comprehension? >> So if you say best way, how, that to me would be a tough one to evaluate. How do you measure that as an outcome? >> So, typically our outcome measure is performance on a comprehension test. So a learner has to learn a set of information. And depending on the experiment, that might vary. And then we're typically interested in, what is the immediate impact on their comprehension. So how do they perform on the test. Right away. >> Right away. >> Okay. >> But we're also interested in the long term, and typically what we mean by the long term is one week. >> Okay. >> So one week later we'll test, give them a different test, and see how much do they actually maintain of that new learned information over that one week. Ideally, we'd like to do that over even longer periods. >> Right. >> But in running experiment, it's typically difficult to get people to come back after months or, or even a year. >> Fair enough. Now, given that, what are the sort of things you're changing up in your experiments? >> Well we have several different lines of research. One line is on multimedia learning. And what I mean by that is, what is the best way, for example, to design PowerPoint slides that will lead to the most effective learning as measured by comprehension? >> Sure. >> So, the tip, in one type of experiment, for example, we might compare, how do learners in three different groups learn the same information? So, all three groups might listen to the same audio track of narration. So that they're getting the same lecture. >> So that factor is constant. >> That factor is constant. >> That factor is constant. In one group, they might be looking at slides that are, are very typical, where they're very text heavy. So, lots of bullet points of text. And that's something that many people are really used to. Another condition might have very minimal text strong visuals that illustrate the main point that's being made on the slide. >> Mm-hm. >> And the third condition as a control might be, nothing, they're just staring at a blank screen. >> Okay. >> And then all three groups will be tested for their comprehension on an immediate test and then one week later. >> Right. Okay so, type of slides is one. Any other factors that you would be investigating in those? >> In those types of experiments? >> Yeah, or in other research? >> okay. So, in other research lines one of the things that we're looking at is called transfer of knowledge. So typically, when students are learning one type of concept or information they might try to memorize that particular concept. Then on a test, they're given an application. >> Okay. >> And that sort of stretches them. So it's not something you can just memorize. >> Right. >> And one of the things that we're looking at is, how do we help people get from memorizing a fact to actually applying it to an application question that they've never seen before. >> Right. >> So this transition is called transfer. >> Right? And that's why we're looking. >> Yeah. That's right. And there are many different ways that we're looking at to try to help this transfer process. So for examples students, when they're learning statistics [COUGH] one of the challenges is that the first week, you might be learning about t-tests. Right. >> And then all your practice problems, not surprisingly are about t-tests. >> Sure. >> The answer is t-test and students learn to memorize this formula and then they just plug in the numbers and they get the answer right and then they feel like, I really understand this. >> Right. >> Then the next week they might learn about a Wilcoxon signed-rank test and then not surprisingly all the questions that week, they're all about Wilcoxon signed-rank test. >> Sure. >> And that's the answer. And they learn that formula and they learn to plug it in. >> Right. >> Now the challenge is on the exam, I give you a question and the biggest, biggest part of the problem is, well what sort of statistical test should I do on this? >> Yeah. >> And students have memorized the formula, they know how to, kind of, if they plugged things in, but they don't really know how to make the transfer process through this novel situation. So, on the front end of teaching there's a few different manipulations we looked at. So one thing that we have found that can help is for example, instead of having a blog learning design where every answer during your practice is going to be the t-test. Or the Wilcoxon signed-rank. In a mixed design, some of the practice questions you have, the answer really is the t-test, but some of them are the Wilcoxon signed-rank, so things that you learned in previous weeks are woven into the practice questions, And students were exposed to mixed questions. It's suppose to block questions even though you do the exact same number of questions. >> Yeah. >> In the end, they perform much better in the final exam. >> All right. Okay, that's so interesting. Now, there's, there's another piece of work that you spoken with me about before on the testing effect and the spacing effect- >> Mm-hm. >> In which I found more fascinating, can you tell us a more about that? >> Yes so the testing effect suggests that if you're trying to retain information for the long term- >> Hm-mm. >> Tes-, being tested is superior to restudying. >> Okay, so simple reading of the- >> Yeah, so the way that we do that is imagine that there is this story passage that all the student's have to read and they're told, you're trying to remember this information. You're going to be tested on this maybe a few weeks from now. >> Okay. >> So really try to learn this information as best as you can. And in one group, immediately after they read it, they're given an opportunity to re-study. So go over, read it again. And in the other, in the testing group. Instead having a chance to re-read and re-study. >> Okay. >> They are actually tested on, on, on the passage. >> Right. >> And in the short term the group that re-studies, they actually do a little bit better than the group that's tested. >> Right. >> But in the long term, the group that has opportunities to be tested as opposed to re-studying will outperform the study group. >> Okay, so something during that test is causing the knowledge to be a little bit more solidified. >> [CROSSTALK] Yeah, so this is something that's really up for theoretical interpretation. One of the ideas in cognition is that every time you practice recalling a memory, it strengthens that memory. >> Right. >> And there's evidence at a neurophysiological level, the cognitive level, that this is actually the case. In fact the best time to try to recall information is just when you're about to lose it forever. And repeating this process actually really helps you to consolidate. And testing seems to tap into this process. >> Right. >> And it's important to note that it's not that final test is the exact same test that you've been tested with all along. >> Right. >> It's a completely different test. >> Right. >> So the take home message that I always tell students is that, to incorporate the testing effect into your own study instead of mindlessly reading and rereading and highlighting your notes over and over, and putting in the hours and feeling like you've done a lot of work. Actually do work by testing yourself, and you're even better off by creating your own test questions. >> So we're in a part of McMaster's Campus called the LIVE Lab which was just built in 2014. There's so much going on here. Can you tell us a bit about this building? >> Yeah, I'm really actually personally excited about working in the LIVE Lab. So LIVE Lab stands for Large Interactive Virtual Environment. >> Okay. >> And it's a 100 seat performance theater with virtual acoustics. So one of the exciting things is that you can make this performance theater sound like the smallest room or you can make it sound like a large concert hall or a large, cavernous church. Anything that you want, depending on the needs of your experiment. What's even more exciting is that 30 of these seats are wired so that from the audience members you can directly measure things like EEG. You can measure heart rate, galvanic skin response and in the future we're also going to set up eye tracking and motion capture. So, these are amazing sets of tools that you can use, and so for my own research, I'm interested in teaching and learning. So I look at this performance theatre as this is actually the world's most advanced lecture hall. >> Yeah. >> And imagine different types of manipulations such as, imagine different groups of people are exposed to different PowerPoint slides with the same narration. We typically ask in addition to comprehension tests, we typically ask people, you know, to, to rate their own learning performance or how well. >> [CROSSTALK] On a scale from one to five. >> Yeah, how well do you think these slides are helping you with your learning, or you understanding, or you interest and engagement with the material? And we have to ask them questions on the Likert scale, say one to five, and they just simply put their subjective ratings, and we've done that because it's the only tool we have available, but with the LIVE Lab we can move beyond these subjective scales and actually look at physiological data. So, we can look at things like galvanic skin response and heart rate variability as measures of arousal. We could look at EEG measures as measures of attention. >> Right. >> So, I'm very excited about these new tools that we look at for new dependent variable measures. >> Wow. Those are going to be some good case studies coming out of this. >> I'm very excited. Yeah, great. Thanks, Joe. Appreciate the time. >> Anytime. >> Good luck with that research. >> All right. >> Thanks.
