Welcome back, we're talking about a qualitative synthesis in your systematic review. And this section is specifically concerned with what is in a qualitative synthesis. As I mentioned before, defining a qualitative synthesis is very difficult. I'll do it in sort of a fuzzy way, or maybe it sounds good. But it's harder to do, actually. And that's what we'll spend this section talking about, is how do you actually do it? So, we define a qualitative synthesis as an assessment of the body of evidence that goes beyond factual descriptions or tables. That, for example, simply detail how many studies were assessed, the reasons for excluding studies. The range of study sizes and treatments compared, or the quality of each study as measured by a risk of bias tool. So that's all fine, and you have to include those tables and those descriptions. But that's not the same as a qualitative synthesis. The purpose of a qualitative synthesis is to try to convey your understanding having looked at the data now in it's entirety about how this intervention might be working or not working. Whether an association that you find or don't find really exists, whether it works for a certain group of people perhaps, and under what circumstances it works. So for example, you will have to make number of judgments as you do in your systematic review, and you will have to say how these judgments might have affected your interpretation of the results. I've said before at the beginning of this course, that systematic reviews and meta-analyses are the most difficult research I've ever done, and the reason is all the judgment involved. Well here's where you're going to have to describe what effect these judgments may have made on your interpretation. So for example, the evidence, do you believe it? Do you have any uncertainty about it? Are there certain studies that bother you for a particular reason? For example, the report wasn't very comprehensive, and there were a lot of unclear areas. What about the implications of missing evidence? For example, one study that you include may be missing quite a bit of data from patients who discontinue treatment and dropped out of the study. How do you feel about that? Might it have affected your results, that they just analyze people who finish the study? Probably, how would you handle that? Would you do a sensitivity analysis? What do you think about the interpretation of your findings given this factor? What about the technical methods that they used? Perhaps some of the studies in your systematic review didn't describe allocation concealment or were unmasked. How do you feel that might've affected the outcomes that you assessed? And then finally, you should be talking about the reasonableness of conducting a meta analysis, whether you've decided to do it and why. What might be the downside of believing a summary estimate of effect, particularly for outcomes that may just have very little evidence or evidence where there have been quite a few dropouts? One of the most important areas of a qualitative synthesis is being able to describe patterns in your evidence. Maybe you see that studies in younger people tend to have results that are more dramatic than studies in older people. Or it could be that a sub group analysis of yours defined an initial baseline characteristic as very important in the outcomes. Maybe that's an important pattern that you want to talk about. Would this help you understand underlying science? Would it help you to interpret your findings? You should definitely describe that. Or perhaps you didn't see any patterns, and maybe that will leave you with questions about what you've found. So, this is your chance to be honest about what you think of the data and to help draw the reader's attention to things that you think they should be looking at as well. The idea of the qualitative synthesis is to try to orient the reader. You want to describe the studies. What happened to the participants? What about the interventions, the comparators, and the outcomes? How do the choices that you've made in designing your systematic review affect the findings that you have? How do the differences in the design and execution of these studies potentially influence the findings and results? And how about the execution of the studies? Perhaps some of them were conducted in outpatient setting and others in inpatient settings. Might that have affected the outcomes? What about if there was a coordinating center for a multi-center study for several of the studies and others were single center? Might that have affected the outcomes? I might to choose to look and see whether there were different results by whether there was coordinating centers and multiple clinical centers in the trials that were included, or whether there were single center studies. This may easily have affected the outcomes. These are the types of things the reader expects you to have investigated, and to bring into your qualitative synthesis. Again, the whole purpose of a qualitative synthesis to is to integrate what you've found and to call attention to specific situations, groups, interventions, outcomes, the way the studies were done. So that people who are reading your systematic review can interpret both your findings and the robustness of your meta-analytic results. So here are the things you probably want to describe. First of all, the things I've been mentioning, the clinical and methodological characteristics of your studies, that might be, are they big, are they small? How might the size of the study have influenced the outcomes? Whether subgroups were left out. For example, you could have in your systematic reviews some studies of just older people, leaving out younger people, while other studies include both younger and older people. How might it have influenced the findings to have included just one subgroup of patients? The older people, or just the younger people. What are some of the strengths and limitations of the studies? How might risk of bias or flaws in the design execution have made these results different from one study to the next? And what is the relevance of your findings to populations that are of interest to you? What about the applicability to populations that are important both in this country and other countries? Might poor populations react differently to the intervention than less poor populations? What about the importance of co-interventions, of the setting where the research was conducted, or the outcome measures that were used in the study? For example, were the outcomes of importance to patients or were all-laboratory measurements made in most of the trials? Nowadays we have a lot more interest in patient important outcomes than we have had in the past. And it's worth it to comment on whether these were included in most of the studies included in your systematic review, and meta-analysis. So, in summary you want to describe the nature of the evidence. You want to interpret it. You want to evaluate its strengths and weaknesses and come to a conclusion. This happens before your quantitative synthesis, if you do one. But the conclusion that you're attempting to reach is, should I combine these studies quantitatively? If so, why do I think it's all right? And if not, why not? We've talked about the Institute of Medicine standards for doing a systematic review, and in fact the book of their standards is one of the recommended texts for this course. If you go to standard 4.2, you'll see the standard on conducting a qualitative synthesis, and there are five different items outlined there for how to do this. Again, it's pretty vague just as I've been somewhat vague. It depends on the systematic review that you're doing. But sometimes guides like this can be of help. And if you think it might, definitely look it up and see what the chapter has to say. So what are the take-home messages? Systematic reviews are integrative syntheses. So you want to integrate all the information that's out there that addresses the question that you're interested in. You will systematically assemble the evidence, and you'll critically appraise it. But you also want to think about the qualitative nature of the evidence as well as the quantitative nature. The quantitative synthesis is not always something that you want to do in a systematic review. It's not always warranted. Maybe the quality of the studies that you wish to include isn't good enough. Maybe the results that each study found are all over the place in terms of results. Sometimes they show a beneficial effect of the intervention, sometimes they don't. What does that mean? When that happens, you probably don't want to do a quantitative synthesis because something's going on. These results are just too different. Well, we use the qualitative synthesis to talk about that. Why are these results so disparate? What might be the explanation? Or, if the results are very similar, this is encouraging you to do a systematic review, and you might want to talk about that as well. In the next section, I'm going to give you a few examples, or show where you can find them. I'm actually not going to talk a lot about them, because that could be boring. But I'm going to tell you where you could find them.
