Hello. Welcome to the Coursera course on designing, running, and analyzing experiments in the interaction design specialization. My name is Jacob Wobbrock, and I'm an Associate Professor in the Information School at the University of Washington. I'm also, by courtesy, in the department of computer science and engineering there. I have degrees in cognitive and computer science from Stanford University and a PhD in human computer interaction from Carnegie Mellon University. But I wasn't always an academic. Before I was an academic, I was perhaps much like you. I was a designer, I still am a designer. I worked in Silicon Valley as a user interface engineer and I also worked as an interaction designer for a handful of years. I also most recently founded a company called Answer Dash out of some of our research out of the University of Washington. And as part of my role there for about three years as CEO, I was heavily involved in the design of our first product. So I have spent a lot of time as a practicing designer, as well as, an academic who studies and uses design in my research. So I've learned through all this that intuition and experience are, of course, invaluable to any designer and we all rely on that a great deal. As part of this specialization you're learning to build those design skills, that intuition and that experience. But, designs also benefit from data and that data often can come from experiments. Experiments allow us to collect data to quantitatively analyze our designs to see if they perform, in fact, as we hope they do. This course is about how to design, run and analyze experiments so that you can better understand your designs, and improve them with data so that you can also gain confidence that your designs are performing as advertised. In this course you'll learn how to design and run experiments, how to pick the right analysis approach for the data that you collect. How to carry out that analysis in the R programming language. A programming language dedicated to data analysis. You'll learn how to interpret your results, and the results of others, when you read other studies in the field. And you'll learn how to report those results so that you're using the correct form when you put them in your papers and reports. You don't need a background in R, you just need R, we're going to use version 3.3.3 and R studio but everything about that language we'll cover in the course. So, that's some of what you'll learn. What this course is not is also important. This course is not a math course, it's not a statistics course and it's certainly not an R programming language course. This course will focus instead on conceptual understandings of what kinds of analyses to use, from certain common forms of experiments that we need in our field of interaction design and HCI. And we'll give you recipes, in the R programming language. I have a terminal here that I'll use that has R code, and we have a handful of data sets that I've designed to illustrate the kinds of analyses we want to do. And you'll get all of that so that you know exactly how to map what we do here to your own studies. That will allow you to understand your designs better and improve them based on the results you get from your studies. So why do we do experiments? In a nutshell, we use data from experiments to predominantly understand differences. And those differences in HCI and interaction design often come in a couple forms. We might want to know differences in performance. Human performance, how do humans interact with your design? How fast are they? How many errors do they make? How do they feel about the design? Those might be captured in preferences. Performance and preferences cover a great deal of the kinds of things we'll be looking at in this course. Experiments may be exploratory. Or they may be more hypothesis driven. Exploratory studies may say, I'm not sure if there's a difference I care about here, let's take a look. Whereas hypothesis might be that you believe there could be a difference and your seeking to confirm of refute that difference. Now you might say well I did experiments in high school or middle school and we measured things in biology class or chemistry or maybe physics and we saw differences all the time. It's likely that at that time you were comparing means. And when you compare means, you're on your way towards looking for differences, but you're not quite all the way there. Let me illustrate why. Let's say we have histograms of a set of scores, and the number of times those scores occurred. The frequency of them. Let's say we're measuring something whose scores look like this. Maybe this is a words per minute in a text entry experiment for a certain keyboard. Let's say another keyboard we're looking at looks like that. Well, the means are here and here. They're clearly different in terms of their position on this graph. Does that mean we have a reliable or statistically significant difference between these two things? It depends. What does it depend on? It depends not just on the means but on the spread around those means. How much overlap is there between these two things? For example, what if the comparison was between this first yellow curve and this curve? Is there more of a difference or less of a difference likely here than here? Those are the kinds of questions that statistical analyses can help us answer. In the end for designers, statistical analyses are about creating confidence based on evidence that your design is what you think it is. We'll organize this course around experiments. Common forms that a lot of studies, and designs take. For each we'll discuss designing them, running them, and analyzing them, and we'll cover, for example, things like user preferences. Which design does a user prefer? We'll cover A B testing of websites. For example, which website gets the most page views? We'll cover task completion times in authoring tools. For example, in programming tools with various languages, or different development environments, which types of benchmark programming tasks take shorter or longer? We'll cover times and error counts. For example, in finding contacts in a smartphone contact manager using scrolling, searching, or voice. We'll cover speeds and error rates in text entry experiments, for example using two different keyboards, perhaps while sitting, standing, or walking. In all these things, we'll see common experiment design patterns that you can replicate for your own work. Along the way you'll also learn how to analyze time, speeds and error rates, analyze subjective feedback on Likert scales, which are when someone marks say one to seven on a scale for how much they like something or how effortful something was. You'll also learn how to counter balance studies and test for order effects to make sure that you're not confounding your results by the order that you present the different conditions in to your subjects. Along the way we'll also work our way through a table that tells you for a certain kind of experiment design, what analyses typically are available to you. And this table will help further your conceptual understanding and give you a place to look in terms of looking for recipes in R for how to do your analyses. Then you can teach yourself further after the course if you'd like to by going deeper into these analyses in this table. Let me show you what that table looks like when it's all put together and filled out. Here you can see at the top a Test of Proportions table. We'll start there. We'll look at user preferences in various forms. Then below that you see an Analyses of Variance table, which is a table showing how you find differences much like these bell curves we just drew on the screen. There's a variety of different analyses available to us based on a number of factors that the table shows. And that helps guide you to the right analysis for your data. Now let's go back and dive into our first experiment. Where we'll discuss user preferences and the design of an experiment to elicit those and the analysis required to look for differences that are significant and meaningful.
