In this module, we're going to talk about non-probability sampling, especially convenience and purpose of samples. In the last module, we talked about probability sampling and how for traditional survey researchers that's often the gold standard of how you conduct a survey to represent populations well. While in UX we may depend on probability samples for some types of research questions, in reality in UX we often are fine with non-probability samples in order to answer the questions we have for the people we're trying to inform. So, what we'll learn about in this module are just the different types of non-probability sampling techniques. So, as a reminder in a non-probability sample, you have this population represented here by the gray circle, and then what you do is you take samples out of that population represented by the blue circle. In strong methods, any one of these small blue dots should represent the whole population. That would be a randomly sampled probability distribution of the population. However in reality often, we don't get access to a portion of the population, we might miss them systematically. So, our blue dot wouldn't represent anything over on the other side of the line. It's often okay that we don't represent that group depending on what our research question is. This goes back to some of our earlier discussions very beginning of this lecture series, it's really important to have the right research question framed that's going to shape whether it's okay to miss that group over there or whether it's not. So, let's talk through some different types of non-probability samples and how we would access those samples and some examples. I think the most common example that people think of of non-probability and the one that you see most commonly I think out in the world is what's known as a convenience sample. At the most extreme, convenient sampling is asking anyone who's willing to participate in a survey to please participate. There's lots of ways people do that. For instance, at the university here it's very common that students will send to a list of all the other students and faculty a request to participate in the survey. The reason why that's a non-probability sample is because not everybody has an equal chance of being represented in that population depending on what population they're trying to represent. So, just targeting students at a university and faculty at the university, if they're trying to generalize to a broader human experience is not going to be very representative as it. Sometimes with these convenient samples, you can be much more targeted. You can focus on specific populations, or sub-populations, or just groups of people you're interested in knowing more about. A good example of I think a convenience sample out in the world is the subreddit. If you don't know it reddit.com is a very large news and discussion site. They have multiple subreddits which are basically pages within the site where people share information and ask for help. This subreddit is called sample size, and it's basically an ongoing stream of requests for help with surveys. What's nice is they'll often ask for help with surveys and then also share results, but this is a convenient sample. What they're doing is they're going to anybody who happens to look at this subreddit and asking them to participate in the survey. They get some great responses, they get a lot of people who respond to these surveys. It's a good way to boost the number of responses that you get, but unfortunately, is not going to be representative of the population. Sometimes depending on your research question of course, that doesn't matter as much. Here's some other examples of convenience samples that might be very common to you. A professor asks all students in a class to participate in a survey. That's very common here at universities especially, you might ask all the people in your office to participate in a survey. I have a student who interned at a company this summer and they asked all the interns to participate in a survey. Unless they've defined their population as interns, then those interns of course are going to be very different than the rest of the world. So, these convenient samples can be fine, it's just you have to be very careful to understand in what ways they represent and don't represent your population. Some other really common ways that people collect convenience samples, marketers standing on a street corner asking for people's opinions about a product. Flyers posted at a doctor's office or a pole on the front page of a website. All of these are hugely common, but each of them of course limits the type of insights you're going to get. So, for instance, flyers posted in a doctor's office is a much more targeted sample than you might get that if you just posted it to a website, but you're still really depending on people to notice it, choose to do that survey, and the people who choose to are going to be very different than those who don't. A form of convenience sampling that people often tried is called snowball sampling. With snowball sampling what happens is you ask respondent who has agreed to answer your survey to send the survey to people in their social networks. Now, this can be great for increasing your total number of responses because basically people will trust their friend to answer surveys even if they don't trust you as a researcher, but it can also lead to what's known as homophily. Homophily in this case really just means that all these people who are friends with one another are remarkably like one another. It's actually a characteristic of human nature that social groups tend to have a lot of the same characteristics. That's not bad necessarily, and there are lots of instances where you might use snowball sampling because you are interested in the social connections between people. So, an example of snowball sampling is solicitations on social media for survey responses. So, I've had a lot of friends who will post on their Facebook or on their Twitter to please answer the survey questions and respond to our questions on the UX survey, that can get some great responses but that's a good example of a snowball sample, depending on their friends. Often they'll ask friends to share that and connect to that friends of friends network. There are some instances when what you want is the shape of the social network. I did research awhile ago where we are interested in Ponzi schemes, and Ponzi schemes are often dependent on relationships between people. So, there we use snowball sampling to get to all the people who had participated in this particular scam that they had participated in. Another example as we were doing work on Facebook and how people connected to one another on Facebook and got value from those connections. In those cases, we were really interested in reciprocal relationships, and we used people who responded to our initial survey and snowball sample to get to their friends network because those were the questions that we had. Another common set of non-probability samples that you're often going to be quite invested in are what's known as purposive sampling. With purposive sampling, what we're doing is we're trying to target a very specific group of people to figure out what their experiences with something were. This is sometimes called selective or subjective sampling, and it's when proportionality when the percentage of these people's experiences compared to the general population isn't important, then it's okay to use purposive sampling to access those people. Now remember, probability sampling is about representing the whole population. Purposive sampling can be really good for just getting at the experiences of a targeted group. There are lots of different examples of specific types of purposive sampling that might be useful for you. So, one example is maximum variation, it's called in the literature. This basically means, when you're really trying to just look at what are the strong differences between two fairly different groups? So, in the example here, Akshaya wants to collect preferences from people of several different age groups. She doesn't need them to represent the proportion of those ages in the population, she just wants to know how younger adults differ on a measure between middle and older aged adults. Say she is doing a paper prototype and wants to share that online and she's trying to get just how these different age groups react. That's an okay time to use purposive sampling. Another example of purposive sampling is critical case. Now critical here doesn't mean in terms of an emergency, it just means that is a specific case that we're trying to sample on. So, as an example, Nico asked people who use a rating system on his site questions about that experience. Participation in the survey is triggered by leaving a rating on the website. In this case, having an event happen could mean that Nico wants to ask surveys of those people in that specific group. He's just trying to elicit from a specific group of people not the entire population what their experiences were. We see lots of great examples of these critical case purposive sampling taking place in the world. Maybe I'm interested in just people who shopped at a store from during a one-week period, or maybe I'm interested in how women who experience this shopping cart tool on my website differ from men. In that case, I don't need to represent necessarily the population of women, I just want differences between two strong groups. A good example of a company that uses purposive sampling very effectively as ForeSee. ForeSee you've probably seen in some of your web browsing. They basically use purposive sampling to elicit user feedback about web experiences, and you often see these mini surveys pop up as part of a critical case of purposive sampling. The critical case in this case is visiting a particular page on a particular website that triggers this small survey, and that goes into a customer database that allows them to shape and characterize what the insights of their customers are. So, in summary, non-probability sampling can work really well for certain types of survey data collection. If you're not trying to show what the proportion of the population thinks this or believes that to be, and you're trying to represent specific cases, specific groups of people, or just to show overall sentiment about something, non-probability samples can be an effective way to collect data. In the next module, we're going to talk about another type of non-probability sampling which is the panel, which has specific characteristics all of its own.
