One of the most critical and challenging parts of designing a user test is selecting and developing the tasks that you will ask users to perform. So, what do we mean by a task? Well it's just the instructions that you're going to give your participants in order to have them try to accomplish some goal using the product. So for example, if it was Amazon you might give the instructions, buy a coffee maker that costs less than $100 and makes at least 32 ounces of coffee. Then you would see how easy or hard it is for them to accomplish that and you'd be able to tell whether they were successful at buying a coffee maker that met those criteria. Or you might ask them to enter a review for the most recent book that they read and you would see how easy and how difficult it is and where they encounter stumbling blocks and where the system supports them particularly well. So how do we go about choosing the tasks that we're going to have for our participants perform during our user test? Well first, we need to ask, what is the purpose of this test? What are we trying to find out from this test in the first place? The general purpose of a usability test is to see if user group X or the users that have a certain characteristic or a behavior, or attitude can use system Y to do activity Z. So, you have to be very clear about who your users are and what it is that you're interested in seeing them do in your test. For example, we might be interested in the question, can experienced online shoppers use Amazon to find and purchase household items? Here we've identified a clear user group, experienced online shoppers, the system is Amazon and the activity is purchasing household items. Or we might ask can people planning a wedding use Amazon's Wedding Registry to create and share a registry with family and friends? So here the group is people planning a wedding, the system is Amazon's Wedding Registry, and the activity is creating and sharing a registry with family and friends. Once you've identified your user group and you're clear about the activity that you're interested in, you want to brainstorm specific actions that your users would perform within the scope of that activity. So, for the example of experience shoppers purchasing household items we might brainstorm things like purchasing a lamp or comparing different coffee makers for purchase or purchasing a specific brand of something like a cleanser for stainless steel appliances. To take the Wedding Registry example, we might think about specific actions like comparing silverware sets or adding a specific appliance like a white Vitamix blender or adding a suitcase set with particular criteria like a particular size, color or material to the registry. Through brainstorming, you should be able to generate lots of tasks that would be good examples of things that people might do when performing the activity that you're interested in. So what you want to do next is choose a set of representative tasks. A lot of tasks that you come up with, are going to be repetitive they're going to basically be testing the same parts of the system and so on and so forth. So, you want to pick a representative set of tasks and then you want to run through them yourself. You want to do this because you want to see how long it takes to get through them. It should take on average about 30-45 minutes to complete all of the tasks. That's a reasonable amount of time to ask a user test participant to perform tasks that you assign them. By running through the tasks yourself, you can figure out whether it's easy to tell what success would look like and you want to pick tasks where you can clearly identify what it would look like to succeed in performing that task. Once you've chosen your high level tasks like choosing a lamp or comparing silverware sets, you're going to want to refine it, so that you can make the task as realistic and verifiable as possible. So by realistic, we mean something that users would actually do. To take the example of choose a lamp, that's actually not a good example of an action that a user would perform or at least it's not specified to the level that is necessary. Realistically, lamps have a purpose, so nobody would come to a site just to pick out a lamp. They would be picking out a lamp for either decoration or for illuminating a room for reading. They would be picking out a lamp for a particular room like the living room versus the bedroom and all of those would actually inform the specific way that that action would be performed. By making a task verifiable, you want to make sure that you can identify specifically what it would mean to be successful at that task. To take the example of choosing a lamp again, users bring a lot of unstated preferences to such a task. So in order for a user to be successful at choosing a reading lamp for their bedroom for example, they would actually need to find a lamp that met preferences like the material that it's made out of. They might prefer wood, versus steel, versus plastic, versus glass. They might have constraints around the brightness of the lamp. They might want a lamp that is particularly bright or not very bright. They might be preferences around whether it has a lamp shade and so on and so forth. In order to make the task verifiable, in order to make the task something that you will be able to tell whether participants succeeded or not, you're going to need to make these preferences explicit, so that all of your users are looking for essentially the same thing and not bringing their own preferences to bear. By running through the task yourself, you can find specific outcomes that you could use to verify whether the task was successful. So, for the choosing a lamp example, we might find this particular lamp and say, design the tasks such that success would be finding this particular lamp. We could do that by looking at well what are the things that make this lamp unique among the other possibilities that are available on Amazon. Well, it has a metal base, it has a lamp shade. This particular one has a three-stage dimmer. It supports light bulbs up to 40 watts and it costs less than $20. So we could make those the criteria that are the preferences that the user is bringing to the task such that to satisfy those preferences, this is the lamp that they would find. So, our refined task might look like this. You are looking for a lamp to go on your bedside table. You want the lamp to be dimmable, meaning that it can put out less than full brightness. You want it to have a lamp shade and you want it to not be made out of plastic. So wood, ceramic, glass or metal are all okay. It should be able to support at least a 40 watt bulb, so that you can use it for reading and you don't want to spend more than $20. Find such a lamp and put it in your cart. Determine how much it will cost if you select the cheapest shipping option and when it will arrive at your house. That last bit is there to help you verify whether the task was successfully completed or not. There should be a correct answer to how much it will cost and when it will arrive at the person's house. The task example that I just gave is an example of a closed-ended task or a task that has a single correct solution. Sometimes in a user test, you will want to include open-ended tasks as well. Sometimes a non-verifiable task can be useful. It can help you learn how users think about a task when unconstrained. This can be particularly useful if you're not entirely clear exactly how users are going to approach the tasks that you're interested in. An example of an open-ended task might be, find a lamp that you would like for your bedroom. So here, we're allowing the user to supply their own preferences rather than giving them constraints that they need to meet. The advantage of open-ended tasks are that you can see what preferences and constraints emerge and how users explore the site. It can be a bit more natural and you can learn a bit more about how your users think about things. The disadvantage is that you can't really tell whether somebody was successful or not, other than whether they feel like they were successful. You also can't measure things like the time it takes to complete a task, the number of errors made, and so on and so forth because you don't have a correct answer that you can compare what they do against. Another disadvantage is that the value of such tasks can be highly variable. Some users will not be able to come up with a motivation to accomplish the task and they'll sort of satisfies and do a very basic version of it, and other users will take it very, very seriously and they'll go into great depth, and you can't really predict in advance what kind of response you're going to get from users. Open-ended tasks can be valuable in user testing. However, closed-ended tasks are going to generally give you more value for the effort that you put into them. So, you want to focus most of your time and effort on designing and testing with closed-ended tasks that have a clear success criteria, so that you can assess how your system is performing and find the specific issues that are keeping people from succeeding at using your system. Once you've selected your tasks and determined how you will verify that they've been successfully completed, you need to worry about how specifically to present the tasks to your users. Well one of the things you need to be careful about is not leading the witness and what I mean by that is not giving away information in the wording of the task that you present that will give clues about how to accomplish the task. Here's an example. If we were looking at Amazon and you were to give the task, "Put three books in your shopping cart then purchase them using standard shipping." This would actually give your participant some clues about what it is you wanted them to do it and how you wanted them to do it. So by using terms like shopping cart and standard shipping you're actually using terms that are used in the interface. In fact they're visible on the screen at certain points and it makes it very easy for your participant to just look at the instructions that you gave them, look at the screen, do a one to one mapping and accomplish the task in a way that might be easier than it would be for them naturally if they came with a different task in mind or a different way of thinking about it. So, what you wanna do instead is you want to try to use relatively neutral terms that don't use the terms that appear in the interface. So you might for example say, "Choose three books and buy them, making sure they can get here by next Wednesday." So here we haven't said anything about the shopping cart, we haven't said anything about the shipping option but we've given a task that's probably more realistic for somebody who's coming with a goal that I need to buy certain books and buy them by this a particular time. So, we talked about picking individual tasks but you also need to think about how the different tasks that you're going to use go together. So when thinking about the task set or the set of tasks that you're going to use, a couple of things you might keep in mind. First of all, you usually want the tasks to progress from easier to harder and there's a couple of reasons for that. The main one is that it can be very stressful actually for people to participate in user tests. They don't necessarily know what's being asked of them they feel anxious about being watched while they're doing something that is unfamiliar to them, and it's nice to give them some success early to help them relax and help them get more into it. To wrap up, tasks that you design for user testing should be relevant to the testing goals, they should be realistic, they should be verifiable and they should be worded in ways that are not leading to the participant and don't give away how you want them to perform the task. The task sets should be carefully designed as well. By being careful and thorough in the design of your tasks you can make sure that you get the maximum value out of the user test that you perform.
