[MUSIC] I'm back here with Robb Olsen to talk about testing prototypes with users. Now, we'd like to start by role-playing a little bit of the sort of conversation that one might or might not have with users. So let's do that. So Robb, tell me, how are you enjoying being here in Evanston and at Northwestern? >> It's a fantastic experience, my wife and I have an apartment in a new building on the 14th floor. >> That's so cool. That reminds me of the time that I was able to teach a class in Barcelona. And I was able to take my entire family, and I taught on a really high floor of a building that actually looked down over the stadium where Barca plays, it was- >> Ed, I bet that was really cool. >> Yeah. >> But that's not the way we do it in user research. >> No, it isn't, is it? No, no, actually, Robb, say a little bit about the sort of discourse that we like to have with users. >> So it's important to ask questions that are kind of get to know you questions, and you did a wonderful job of that in this first bit where we just worked role-play. >> Mm-hm. >> However, then you cut me off and started talking about a similar experience that you had had. >> Right. >> Now of course, that's normally how people interact socially. I'll say something, you'll relate to it, you'll tell me how you relate to it. That's great for discussion among friends or at a party, but it's not the way we do it with users. >> Okay, so with users, of course, we really want to get them being the ones talking. We want to elicit more, so let's try this again. >> All right, let's go. >> Let's see if can do it, I'll try to do a little better. So Robb, tell me, how are you enjoying being here in Evanston and at Northwestern? >> Oh, it's great. My wife and I are having a wonderful urban living experience. We're in a new apartment building up on the 14th floor, and we've got this great view of the lake to the east and Evanston to the north. >> Robb, I've never actually lived on the 14th floor, what's that like? >> It's really fun. We're up above the city and kind of looking down, and sometimes I feel like the city is made up of all these toy cars and people. And I'm watching everything go by, and we're between these two train tracks, and I hear them come and go. The L has this really cool sound that I call it the click clack. >> All right, so I think we did a little better that time around. I was able to sort of control my urges and ask you to tell me more about your experience. And so that's really important when you're interacting with users, but there's a lot more to it than that, isn't there? >> Absolutely, absolutely. So when you first are engaging with a user that you're going to do research with, put them at ease and take the attitude that they are the teacher and I am the student. So that first question you asked me was great, for a get to know you kind of question. And then, I will usually tell the person I'm doing the research with that they are the teacher. And that opens a wonderful door of richness in the interactions. So they flip a little bit from normal social discourse to actually trying to help you understand their life, and for example, how well your prototypes are working in the research you're about to do. Okay, so you put them at ease, and then the next step, I like to observe their behavior. So I'll have a defined behavior I want to observe, if I'm working on Tide, it's doing laundry, and again, that's a very natural thing for them to do. The one thing you've gotta do is make sure they understand when you're coming to do the research with them, that you want them to be ready to do this task. So that's the kind of the ethnographic approach. >> So let me understand, so this this work is ethnographic in the sense that partly what you're doing is trying to sort of go into the user's world and understand that. >> Yep. >> A little bit like Margaret Mead went to the islands in the South Pacific and really just tried to understand the indigenous people, so it's in that- >> Excellent example. >> Okay. But of course, at some point, you need to perturb that because you've got new prototypes to test. >> Absolutely, and that's where I switch to a kind of behavioral psychology approach. >> Okay. >> So I think of this as the prototypes are the stimulus, the study subject is the user, and the data coming out of that is how they respond to the stimulus. So when they use what they currently use, you observe that. And then they use one of your prototypes, you're looking for the differences in outcomes and behaviors from that. >> Interesting, okay, now do you typically just take one prototype? I guess in my experience, I think that people are able to sort of compare and contrast things better than sort of give absolute judgments. So I'll often take multiple prototypes, how about yourself? >> I do about as well, absolutely. Often, 4 or 5, I have actually constructed experiments where we've had as many as 20 different prototypes. >> [LAUGH] Okay. >> Now in an experiment like that, the behavior is really well-defined and very short, so that the user can get through that number of interactions with your prototypes, but at least three or four. What three or four, well, it's the ones where you as a designer and perhaps your team have different points of view on what might work best. Perfect, go take those, turn those hypotheses into testable prototypes, take those to the user. And be data-based in your decision making about which of those approaches works best. >> Well, let's talk a little bit more about data. So you're out there in the field, in the proper context, working with the user to test these different prototypes. What sort of data do you typically collect? >> So, first and foremost, we video. >> Okay. >> And video is the data of a user interaction. >> Mm-hm. >> And so you need to be kind of robust about it, you need to, first of all, get the user to agree it's okay to video. And let them know how you're going to use the video, it's not for posting on YouTube, it's for research purposes. And then make sure that your team is structured such that, somebody can be doing the interaction with the user, somebody can be capturing the video, and ideally, somebody taking notes along the way. For example, when some really rich insight happens, he wants somebody saying, gem, right here and kind of noting the time, so you can go back and find in that video, and that kind of brings me to analysis. If videos, the data needs to be treated with the same respect as numerical data from an analysis standpoint too. A really rich thing you can do is go talk to a whole sequence of users. And then, instead of listening to user one all the way through, then user two all the way through, cut it the other way. Take the first behavior you ask the user to do, and put that back-to-back across users. And then, prototype one back-to-back across users, and prototype two back-to-back across users. And a whole new rich set of insights comes out of that, that really helps you with the design process. >> Wonderful. So to recap, the idea is that you go into the user's context, you get to know them a bit, you put them at ease, and you understand how they solve the problem today. And then you review not one but multiple prototypes, and you attempt to record everything that's going on, in notes, in video, and even pictures. And of course, you do all this humbly, treating the user always as the teacher, even when they're using your prototypes. And you keep asking that wonderful question, tell me more. So Robb, thanks so much for doing this, I think it was a terrific conversation. >> Thanks Ed, I've really enjoyed it, and I hope we've helped some of these students along in their design and prototyping journey. >> Absolutely. [MUSIC]
