So far we've considered analyses of variance and nonparametric tests for between subject factors only, where each subject only gets one level or one treatment of all the possible levels of a factor. Now let's consider within subjects factors and here's the scenario we'll work from. Let's say 20 subject interact with a smartphone contacts manager to find a set of contacts that they're assigned to look up and they do this initially in two ways. These 20 subjects each use two techniques. One is searching, typing text and looking through the context manager that way. And they other technique they use is scrolling, just scrolling a list and visually looking for contacts in the list. Later we'll add a third technique they'll use, which would be voice search, speaking the name of contacts and trying to look them up that way. Of course again, as with all our data sets, this is just fictitious data for our use in this course. The things we measure will be total time and the number of selection errors. So we're measuring both time to find a contact and the number of errors made, where someone selects the wrong contact. Perhaps moving quickly and tapping the wrong name, or other kinds of errors that may come up. Subjects will also rate their effort on a 1-7 scale, what's called a Likert scale. So we get some subjective feedback about how much effort they think each technique takes. So, to put this down, we have 20 subjects, we have three techniques, initially two then three, and they are Search, Scroll, and later Voice. And we have the measures that we want are the time to find the context, the number of errors made, and the effort on a 1-7, what's called a Likert type scale. These are getting a little bit more interesting than what we've looked at before. But we're able to take on more complicated analyses because we have more sophisticated analyses at our disposal. Okay so, we can ask ourselves with this one factor called technique with three levels or two levels initially, it's a nominal factor or categorical as we've seen before. We have time which is a dependent variable or response and time is a numeric result. We have errors, also numeric. And effort, this would be called ordinal measure because a 1-7 scale while is a number, is an ordered scale. And the gap conceptually for a person between say 2 and 3 may not be how they perceive the gap between say 6 and 7 or 5 and 6. Those gaps may be bigger or smaller, they're more of a subjective concept. So that's called an ordinal variable where you know there's an order but you don't know the distance between each step in that order. Now, is technique a within or between subjects variable? Well, we said each of the 20 subjects does all three techniques, so therefore, it would be a within subjects variable or within subjects factor. These are also called repeated measures. And you'll see those terms used interchangeably, it's important to understand they're the same thing. Now, let's ask ourselves, when should we use within subjects variables, versus between subjects variables? What would be the considerations there? Well, the short answer is, you should use it within subject's variable whenever you can. Why is that? Well, for one thing, it takes many fewer subjects to get the same amount of data. We would need 60 subjects if each subject only did one of the techniques. But we'd need 20 if they each do all three. So we have more data from fewer subjects we have to recruit. It's also preferable because we want to remember that experiments are designed to detect differences. Well, what are the differences that we're looking for here between the three different techniques? We want to know in our measures if those three techniques perform differently. And when we're trying to detect differences, we're doing that against a backdrop of natural variation, measurement error, and general noise. So anything that reduces the variance in our measurements is a good thing. Leaving to stand out, hopefully, the variance or differences in the things we care about. And so, when we have within subject studies, the variance is less than if we have between subject studies, because you are more like you than you are like anyone else. Every subject is more like themselves than they are anyone else. And so, that reduces individual differences and the variance that rises as a result. But there's one very important challenge with within subjects factors, and that is the potential for carry over effects. Because each subject. Because each subject, you really look out for that. Because each subject does all three. We have to be careful that the order that they do them in is not causing them to perform differently then they would if they only did one each, and we had a between subject study. We have to be careful that we're not introducing a form of confounds. We've talked about confounds before. Now, carryover effects are a major source of confounds and within subjects' studies because the order we present things in can confound the result. So what are the kinds of carry over effects that arise in within subject studies? Examples are fatigue, practice effects, boredom, or skill transfer, where being in search made you better at scrolling, or being in scrolling made you better at voice and we presented them in a certain order. Then we might be in fact differentially affecting future conditions based on prior ones, those are skill transfer effects. How would we test for an order effect? Turns out we can know if they're happening, but we have to encode another variable in our data table. And that variable would be the presentation order of the conditions. So we have technique as a variable but we'd also add to our set the variable order or maybe technique order. And we'd encode that with just say one two or in a case of voice being added, three for which order was the measure taken in, which order was the technique shown in? And then we simply do a test like we would on technique to know the difference between the techniques. We do a test on order 1, 2, or 3, to see if there's a difference and the result just based on the order that things were shown in. And I obviously hope there is not such a difference. That would mean we don't have an order effect. So we want to think ahead to make sure we log that as part of our data. It still leaves open the question though, how are these orders assigned. How are orders one, two, and three actually put together? And for that, we want to go to our counterbalancing schemes. We'll discuss that next.
