Hello everyone. In this demonstration, we're going to illustrate something called the Survey Quality Predictor application. This is an online tool that can be used to evaluate the overall quality of survey questions. Again, this is in the realm of validity to make sure that the questions that we're asking in a given survey makes sense and will provide valid measurements of what it is that we're theoretically interested in. We're going to this online tool here and just demonstrate some features of the tool. This is the Word document that's been provided along with this particular demonstration. Again, we can just walk through these steps to get a sense of how this tool works.. The first thing to do is go to sqp, for Survey Quality Predictor,.upf.edu/. I've included that particular web link here. You see this brings up the homepage for the Survey Quality Predictor. SQP 2.1 is a survey quality prediction system for questions used in survey research. Very important to look at reliability, validity, and quality of the different survey questions that you're using. This is one of several possible tools that one could use to really get a sense of the overall quality of the questions that have been written. When you go to this particular website, you can then click on the green Start button to get started with the tool here. Now, you have different options here. You could add questions to their overall database of questions that they maintain. You could download particular information or questions from the database that they maintain at this particular website. We're going to choose the option, as it says in the Word document, to go to the database, just to look around a little bit and see how this works. This is a database of overall survey questions that have been created in a variety of different languages, in a variety of different countries. Each of these questions has been scored based on statistical models that evaluate the overall quality of the questions and make predictions in terms of the overall validity and quality of the questions that have been written. Many things we can play around with. We're just going to choose the first overall option here for SQP illustration just to illustrate some of these ideas. You can see this extensive list of languages here that you can look through to find what type of language you'd like to see for a given question. I'm going to actually scroll to English here. This is for purposes of the demonstration. You can see there's a small number of questions that pop up here. Under this SQP illustration category, we could choose different categories of questions if we wanted to. Just for illustration, after we click English, we'll click on this very first question, which is measuring satisfaction with the economy. I'm going to click on the link for this particular question. You can see that this pulls up a window that provide some details about the question, the text of the particular question, the different answer options. Most importantly, down at the bottom of this window, you can see what's called an MTMM Estimate. We'll be talking about what that means in the course. Then also an authorized prediction of the overall quality of this particular question. That's what's really cool about this database, is that these questions have all been scored in terms of their overall quality. As we see in the Word document, you can go to the additional link here to see computational details. But essentially in terms of those predictions, a value of one would mean that that question is of the highest possible quality, and a value of zero would mean that that question is of the lowest quality. These predictions are going to be on a scale of 0-1, where again, these predictions have been based on prior statistical models evaluating the overall question text and the number of responses provided. You'll see these types of scores for each of the different questions. That allows you to pick the questions in the database that you feel have the highest quality and are also getting at the essential concepts that you want to measure. Very useful feature there in terms of having these scores available. You can view details at these different links if you'd like to for a given question. Another option is that you could add a new question to the database. You see this green link here on the left-hand side, Add New Question. I can click on that particular option here and then you can see you can add details about the particular study, the language, the text of the question, the different response options and so forth. All of these, I'm not going to go through that in detail right now, but you could add your own survey question to the database and then have it scored and have it retained in the database for others to look at. That's also a very nice feature here in that you can add new material to this database. They keep this database active for other researchers to use. For example, in our prior Survey Monkey example where you designed your own web survey, you could take one of the questions that you asked in that particular survey, add it to this database, get it scored, and evaluate the overall quality of that particular question, considering its text and the different response options. This is a nice example of an online tool free to use, that you can use to evaluate the overall quality of different questions and identify some different existing questions in that database that have already been scored in terms of their overall quality. Again, you can see there's a science behind this in terms of developing these questions, evaluating the ability of respondents to comprehend them, making sure the response options are clear. The Survey Quality Predictor application allows us to assign a score to these different questions and evaluate different questions based on their overall quality. This is an important step when you're assessing the validity of the questions that you're asking in a given design data collection. This is one of several useful tools that are available for evaluating the overall quality of the surveys. Thank you. Play around with this a little bit and we'll continue to talk about issues related to validity. But this will give you a chance to explore one of these online tools for assessing the quality of survey questions.
