So continuing along in this Special Considerations Module. We're going to look next at questionnaire planning and development. There are a number of key principles here that are important, some of which only apply in the survey domain. So, the first thing that is important, and again, remember we're looking at this from the standpoint. That an individual taking this particular data collection instrument participating in this study. They may be taking it on paper, they may be taking it by a web survey But, but, they are not going to have someone in the room with them to ask questions, should they have should, should there be any kind of concern or ambiguity. So, so we need to make sure that we're being very careful with these questions. First concept that, that I would typically try to think about is making sure that each question. Is concise, and then it's targeted to a single concept. So, a bad example here, would be a question that, that asked an individual to. To specify whether they thought this course was well organized and the instructors delivered interesting lectures. That's tricky because there are two questions. So an individual might look at this course and say it was really well organized but I just hated those instructors. So, so they wouldn't know how to answer that question. a better example for that would be, you know, this course was well organized, yes, no. And that the instructors delivered interesting questions, yes, no. So always be careful. Any time you are looking at questions and you see the word and. Or. Be careful, and just kind of look at that and make sure that it's going to make sense, and that you're going to be targeting a single concept. Next topic or next, next sort of best practice would be planning questions that relate directly to answering specific aims for your study. We again, talked about this in, in some of the foundational lectures. Talking about best practices. But, but, I've found in survey type research, where individuals are putting surveys together to ask question to participants. It's extremely extremely important and it's, and it's quite, quite hard to remember. To avoid nice to know questions because you'll not going to be in the room with them. You'll not going to see that respondent fatigue. But be very careful not to just throw in a bunch of questions that really don't point directly to your objectives and specific aims. Because you'll end up with, you know a whole lot of data elements and because of respondent fatigue, you know they may not finish the survey. Or they may not put put as much thought into the answers as they should. So every question should be a meaningful contribution to the end analysis. Carefully plan each word in every question. Again this goes back to. The the concept that nobody's going to be around if they don't understand it. So, so make sure you're thinking about the population. You know if you're, if you're surveying a group of individuals and you think that there's a good chance that they might have an eighth grade education. You know don't, don't come at them with a lot of medical jargon. Make sure that the that, that the terminology is familiar to them, that the concepts are familiar. Make sure that you're thinking about specific wording that they would understand. Simple grammar, non technical,uh, avoid acronyms as much as possible. Unless you know you, you are surveying a group of experts in a field and then you know they'll, they'll. They'll, they'll be fine. But, but for the most part, try to avoid those acronyms. And, and then, finally, try to strive to to make sure the, the language in your questions are neutral and non-biased. A, a bad example here might be a question that said in an average week. How many days do you drink too much alcohol? On the surface, it sounds like a, like an okay question. But when you think about it, you know, there, that's very subjective. You know, to an individual that drinks much alcohol, maybe they would answer a very high number there. And and you know, not think that it was a, sorry, they might answer a low number there because they don't think. You know, really, really what they're consuming is too much alcohol. Whereas if you had an individual that didn't drink much at all, maybe they would have a different threshold. So better to be very specific and just ask, you know, how many, how many alcoholic drinks do you, do you drink in a particular week? Rather than leaving it ambiguous, as, as we've done here. [COUGH]. plan questions of recall very carefully. If you look at the literature, you'll find over and over and over again that when individuals try to remember something that happened yesterday. Or even this morning,uh, in, in all types of different domains, but especially a lot of examples in the nutrition research area here. Yeah, you, you'll find that, that even short-term recollection is quite flawed. and long-term recollection is really, really bad. And, and you know I think most people are typically very honest. I mean most people are actually quite honest, but, but in fact, their memory is quite faulty. So when you're creating surveys to try to. Get them to recall things that are going to answer a specific question or topic for you. Keep this in mind. Short series of related question can help focus on the recent past if, if at all possible. And I like to sort of define a reference time frame that's as short as possible that might be representitive of something we could extrapolate. And, and go on to to assume larger, larger time periods. Such as during the last seven days how many days did you smoke at least one cigarette? Versus asking them a question, you know, you know, on average, over the last five years how many cigarettes did you smoke per day? So, so be very careful on those recall bias, types of questions. grouping questions is generally a good idea. You know, if you're sort of, you're having them think about specific domain areas, it's good to ask those in, in, in one place Rather than sort of having them all over the place because it's might might be easier cognitively to have your participants sort of answer those all at one time. You just have to be a little bit careful and not lead the respondent in the fact that they might answer question ten. a little differently had you not asked question five because they're remembering. Well I answered that question five this way, so I guess I'd better answer this one this way or I seem inconsistent. So grouping is a good thing, just sort of be careful and watch about that leading the respondent. I've seen some researchers. And I've, I've definitely taken some research surveys where researchers will ask an identical question in different parts of the survey. Particularly long surveys. And, and the rationale there is just to make sure that they're paying attention. Making sure that the respondents are paying attention to questions. And, and that they're consistent in their answers. And while, you know, this, this might be a good idea, in, in, some survey research, you know, I'm sure it's, it's a valid concept or people wouldn't do it. I know that personally whenever I find that that happens to me. It really annoys me and so I think you have to be very careful not to annoy the respondents if you're going to, if you're going to take that approach. grouping demographics and sensitive questions at the end of a survey can sometimes increase completion rate. And, and the idea here being that if the first thing I ask or first thing I'm presented with is. Some, some question that's sort of sensitive in nature and I don't feel too bad about answering it. But I'm a little bit uncertain if that's the first thing I see then, then I may not want to go on and finish. my, my, might decide I'm not going to start this survey. Hey, I'm, I'm not sure what I'm getting into. Whereas if you, if you sort of leave those types of question until the end. sometimes, sometimes people will, will feel a little bit more involved in the process and they might You might have a higher likelihood of having them, completed. Of course, you need to, to align things so that the participant can drop out. Or, or, leave, leave the survey at any time if they don't feel comfortable answering those questions. consider both use of open, closed questions. So an open question, might be describe any negative side effects you've, you've, experienced while taking drug A. Where as a closed, type of, question might be. Do you ever feel or have you ever felt nauseous when taking drug A? And, and have that answered with a yes or no. So, you know, open questions are quite flexible. They allow the, respondents to provide answers in, in their own words. They they, they also allow some concepts to come through from participants a, around things that maybe you hadn't thought about answering. the difficult thing about asking open-ended questions is that they're hard to analyze. Now some types of research it, they're absolutely essential. but, but, whenever possible. it's, it's good, I think, to sort of keep, keep things in a closed level. Or at least have most of the questions in a closed level. If you know what you're going to be measuring. if you know what you're going to be asking. Because they're easier to analyze and they really, you know, just sort of streamlines things. But the flip side of that is that it does not allow. Respondents to express all opinions and, and may limit the accuracy. What, what a lot of folks do, and what we try to do on my team is we will, maybe do a pilot survey first where we ask some open ended questions. Based on the answers to those open ended questions we might, we might let that inform the process of. Of fleshing out the rest of the closed questions so you can sort of use that type of approach as well. even with open questions, there are ways that you can analyze it but, but typically, if you're going to analyze the open-ended questions. You're going to have to have somebody to come in that's unbiased. That's that's looking at things in a, in a strictly blinded fashion and allow them to sort of read the answers. And then code those to a, to a certain set of structured variables. and then you can go off and analyze them. So, can be done, but it but you get incomplete answers that way, and it's it's a lot of work. So try to close as much as you can. again I mentioned this just now. You know a lot of times we'll do open questions in pilot survey development. And then as we kind of get get, get the information there from those open survey questions. We might, we might use those to create more closed questions for downstream use. So, This is, this is something you a lot on, bad surveys. It's, It's important to structure the answers so that the respondents always have an opportunity to answer correctly. So, you know, I've seen surveys. I've participated in surveys, where you know, the question would be asked, what is your age in years? 20 to 30, 30 to 40, 40 to 50. This seems pretty good until you start looking at it and you think, well, what if, wait a minute. What if I'm 30 years old? What if I'm 40 years old, you know? Do I answer, if I'm 40, do I answer category two or do I answer category three? So, so be careful and make sure that, that, you're not boxing people in to confuse them by, by, by Not giving them an obvious way to answer the question. it's also a good idea to include choices. not necessarily in the age question but in other types of questions that you might ask. It's a good idea if there's any ambiguity there to allow them other so to please specify type question. As well as I don't know or none of the above options. It's important to make sure the questions and the answers match in syntax. Again this is not such a big deal if we're asking these questions looking into the eyes of our respondents. But you're not there, so if you ask them the questions where the grammar doesn't quite match up. Have you had pain in the last month? That's a perfectly valid question. seems like that would be answered with a yes no but here we've said Never, Seldom, Often, Very often. So it's grammatically close. And you know you might not catch it on the first time through designing your survey instrument. But that inexact match of question and answer pairing will lead to some confusion and at least annoyance. You know, better example in that, in that particular case would be how many days did you experience pain during the last month? It's important to include units of measurement. We've talked about this in, in the past, in, in terms of best principles, but again it's, it's especially important when you're doing survey research. How tall are you? You know am I going to put that in centimeters? Am I going to put it in feet and inches? A better way to ask would be how tall are you in centimeters? consider common question types. And I've got a number of these on, on the screen here. Yes, no, we've already talked about. True, false, we can ask questions. whether we're doing it by paper or electronically multiple choice with only one answer. A Likert scale, which we'll talk about in the next several slides. multiple answers, dates, text boxes where we validate the, the answers or, or text boxes, or, big, big text boxes. Or comment boxes where we don't for open-ended questions. As well as visual analog scales which we'll talk about next. So that's it for this segment, we'll pick it up in the next one.
