When planning a user test, the first question you need to ask yourself is whether a user test is actually what you need? Is it the right tool for answering the questions that you have? User testing is great for understanding how well a design supports users' tasks, does it allow them to perform and accomplish the tasks that you've designed the system to do? It helps you understand how easy it is to learn to use a system, and how efficiently people can use the system once they've learned it. It shows you where people fail, and what causes the failure so that you can go and fix those problems. What user testing is not so good for is, knowing what users need or will find useful. It's not good for predicting how many people will encounter the problems that you identify in a user test. It doesn't really tell you how people will use the system once they have it in their own hands, and they're using it without your supervision. Unfortunately, it doesn't predict whether people will adopt or use your system, and it doesn't predict whether your product will be successful in the marketplace. Understanding what users need is best done by observing what they do now. This is typically done by observing their current practices, and through interviews, and surveys. If you do understand users' needs, and you've gotten them correctly, user testing can tell you if your system meets them, but it can't tell you whether you've gotten the needs right. User testing can find problems with the design, but it can't necessarily tell you how common those problems are going to be in a larger population. This is because user testing sample sizes are generally too small for that kind of prediction. So user testing, definitely tells you that a problem exists, that some portion of the user population will encounter, but you don't know how large that portion is. However, you can argue for the importance or the likely significance of a problem based on your understanding of how representative your user test participants where, the user population needs, and characteristics of the target population. User testing can't really tell you how people are going to use the system once it's released into the wild. In a user test, you're the one providing the tasks, and all you can really see is how people perform those tasks.in the real world, users are going to be the ones that choose the tasks that they perform, and sometimes, people don't do what you expect. You can ask people what they will do, but people are actually pretty bad at predicting their future behavior, and there's a lot of research that backs this up. Analytics are the best way to see what people actually do, but this doesn't come until after a product launch. During design time, it's very difficult to predict exactly what people are going to do with the system once it's released. Predicting adoption and use of a system is even more challenging. Users' decisions to adopt and use a system are complex and take into account a lot of different factors. Usability matters, but so do things like awareness of a product, and the ability to find it. The comparison with competing offerings, and their existing practice like what it is that they do now, and how they do it are also very important as are perceptions of value. So, the weighting of the cost versus benefits of switching to your system or using your system as opposed to a different way of doing things. You might be tempted as part of user testing to ask people if they would use a system, if they'd be likely to adopt it. You might get an answer that you want to hear, but again, people aren't good at knowing their future or predicting their future behavior, so you have to take their answers with a grain of salt. Perceived usefulness and perceived usability are linked to adoption decisions, but they're not sufficient for predicting whether people will adopt a system or not. Finally, predicting whether a product will be successful is incredibly challenging. Decisions around adoption, and use and satisfaction with the process of using a system all make a difference. Usability and usefulness matter a lot, but they're not everything that goes into product success. Other factors such as the business model, the costs of supporting and updating and producing a system, all matter to ultimate product success. But if the usability stinks, success is definitely much less likely, and that is something that we can learn through user testing. So, user testing is a critical research tool, we need it to figure out if a system is successful at supporting the tasks that people need to accomplish. However, we need to use user testing as one tool in a toolbox of UX research tools, along with things like interviews, observations, surveys, focus groups, analytics, AB testing or website optimization, usability inspection, and so forth. It's important to know both the advantages and disadvantages of user testing, so that you can choose the right tool for the right job.
