We discussed the fixed effect model in the previous section, and now we're going to talk about random effects model. We're going to see how random effects model is different from the fixed effect model. And, we're going to contrast those models throughout this section of the lecture. A quick review. Under a fixed effect model, we assume that all of the studies are identical, and the true effect size is exactly the same in all studies. Well, then the question is, how plausible is that assumption? Is the assumption underlying a fixed effect model plausible? What do you think? Well, then, you're going to tell me, we have to look at the studies in trying to figure out whether the studies are similar or different. Exactly, that's what you do. For example, the magnitude of the impact of a educational intervention might vary depending on the class size, the age, and other factors, which are likely to vary from study to study. So let's think about this for a moment,. Who on earth are going to do a study that is exactly the same as a study that has already been done? You may get two identical studies for those drug trials submitted to regulatory agency for approval. For example, for a drug to be licensed in the United States, the drug company have to do two identical trials and submit the results to the FDA. In that case, you may get two identical studies. But for most part, what you're going to have, for your systematic review and meta-analysis, are different studies done by different investigators in different places. We may or may not know for sure whether these characteristics are actually related to the effect size. For example, you may argue that vitamin D studies done in the United States are going to be different from those studies done in Asia. Because some countries may be more closer to equator, such that they will have more sun exposure. So their Vitamin D level, the baseline level in the participants, are different. We have all this knowledge to help us to evaluate the factors that may or may not be the same across studies. But for most time, logic suggests that such factors do exist and will lead to variations to the magnitude of effect. So what are we looking at? We're looking at the characteristics of the study. It could be design characteristics or the characteristics of the study participants for a study setting, for example. And these characteristics may be related to the fact size. So, that's all we care about. We care about these characteristics because they may have effect on the effects size we're going to see in those studies. Here is an example from one of the class projects back in 2011. So this study look at the association between the duration of breastfeeding and the risk of childhood obesity. So the hypothesis is that the duration is associated with the childhood overweight. And here is what the review group found. Our 21 studies were all cohort studies, of which eight were in the US, nine in Europe and four in Asia, Australia or the Middle East. The studies analyzed breastfeeding duration ranges from as little as 0-16 weeks to as much as greater than 12 month. And the sample sizes range from a little over 300 to over 117,000. And the study dropout rates prior to follow up ranges is from 5% to 52%. Again, since all of these studies are brought together in a systematic review of meta-analysis, we are answering a clinical question. Which is, is there any association between the duration of breastfeeding and the risk of childhood obesity? But each study is different from another study. And, we are concerned about whether, for example, the duration for breastfeeding is associated with the association you are going to see, okay? Here is another example. This example examines the risk of ischemic stroke in people with migraine headache. And the authors wrote, as seen in our descriptive tables, there was substantial variation in the sample sizes and characteristics of the research subjects across studies. Subjects were joined from registries, administrative databases, randomized control trial participants, hospitals, and the community. These various source populations would be expected to be associated with different baseline risk of stroke. And the studies also varied in mean age of subjects, which range from 15 to 97 years. But most focused on the 15 to 50 year old range group. Again, the authors are concerned about the age distribution from these studies. So the younger population group will, based on our knowledge, have lower risk of stroke, right, comparing to the older population. And that's why the association you're going to see between the migraine headache and stroke might be different from study to study. Careful qualitative synthesis of the data usually would indicate that there are diversities, clinically and methodologically. And they may lead to variations in the magnitude of the effect size. We call this variation underlying the effects heterogeneity. So heterogeneity refers to the clinical methodological diversities among a set of studies. And what can we do about heterogeneity? Well, If the studies are too different, let's say, we're really comparing apples and oranges, we don't have to do a meta-analysis. Remember, the first slide, I said, meta-analysis is only an optional component of a systematic review. If they don't belong together in a meta-analysis, you don't put them together. You can report the estimates from those individual studies. Or we could seek to explain why the studies are different. So let's say, again, the vitamin D and placebo example. All of the characteristics could look very similar, but the difference is in the vitamin D dose. Well then, we can do additional analysis, trying to figure out if the effect size from vitamin D is associated with the dose. If there are no good explanations, where we cannot figure out a reasonable reason why these effects are different, we could allow for it without explaining it. And the way to allow for it is through a random effects meta-analysis. Under the random effects model, we assume there's a distribution of true effects. Remember the assumption under the fixed effect model? We assume all studies are identical and there's only one true effect. So here, instead, we assume there's a distribution of them. So again, the circles represent the true effect in studies. We're using the same example. There are three studies, so we have three circles, the true effect. And under the random effects model, these three circles no longer coincide with each other. Now there is a distribution. So if you look at the little normal curve underneath the circles, that's the distribution of the true effects size from those three studies. In contrast, if you still remember the plot from the fixed effect model, all the three circles coincide or overlap each other. And again, using the same notation, the observed effect size in Study 3, that Y3 now differs from the true effect in Study 3 by that epsilon. So the error term within that study. But because the study, we're assuming the true effect size follow a distribution, there is one more source of variability, okay? The variability is denoted as vita 3, here. And instead of having only the within study arrow, the true effect in Study 3, the Y3, can now be written as the mu plus the zeta 3 plus the epsilon 3. Again, the Y3, the 0.4 can be written out as using the mean of the distribution of true effects among a population of studies plus the zeta 3, which is the difference from the true effect in Study 3. From the overall true effects plus the error term in that study. Here is another way to look at it. Here, the Y3 can be written as the mu plus zeta 3 plus epsilon 3. Now, the distance between the overall mean, and the observed effect, in any given study, consists of two distinct parts. The true variation in effect sizes, which is the zeta i's. And, the sampling arrow epsilon i's within the study. More generally, the observed effect Yi for any given study can be written as a grand mean plus the deviation of the study's true effect from the grand mean and the sampling error in that study. So now, because we have assumed all the circles, again, going back to the circles which are the true effect in each individual studies, there's a distribution, okay? So that there are two sources of variance. The first source, if we just focus one study, there's within study variance. So the distance from the theta i, the circle, to the Yi, the square, depends on the within study variance. So, the variance is the random errors within that study. That's the within study variance and we have that. We have exactly the same within study variance from our fixed effect model. On top of that, there's another level of variance which is the between-study variance, okay? So the distance form the mu, the triangle, to each theta i, the circles, depends on the variance of the distribution of the true effects across studies. And we call that variance tau square. Because all of the circles, again, the circles on this plot are the true effects in each study because they don't line up together, they don't coincide with each other, there's a distribution. And that distribution is the between-study variance. So under a random effects model, we have to capture both sources of variance,. The within study variance, as well as the between-study variance. So here's a nice contrast, of the fixed effect model and a random effects model. The different assumptions you are making. Again if there's only one thing you are going to take away from this section of the lecture, is this slide. The different assumptions underneath, two different models for meta-analysis. On the left-hand side, all three figures. You have seen all of them and they are the assumptions for the fixed effects model. There's only one assumption and they're showing you three different ways. Under the fixed effect model, we assume all studies share a common true effect size. That's why all the three circles, lying on top of each other, so there's only one identical common effect size. However, under the random effects model, if you look at the figure on the upper right-hand corner, here's a distribution. The three circles no longer align together. There's a distribution of effect size, okay? And because of that distribution, we added one more level of variability, which is the between-study variance. And that's why you have to capture them in your analysis using the two slightly different equations. Under the fixed effect model, Yi equals theta plus your error term within study. However, under the random-effects model, the Yi equals the mu of the grand mean plus the zi. That captures the distance of each circle from that triangle, plus the epsilon i, the error term. Again, the difference is, under the fixed effect model, there's one source of variance. If you look at the last set of figures, right, there's only one source of variance, which is captured by that normal curve for each study. However, under the random effects model, we actually have one more layer, which is the variance between studies. That's why you have four normal curves instead of three, which still have that within-study variance. But however, underneath the last plot, that little curve shows you the variability. The distribution of the true effect size. That's your between-study variance. Now let's take a pause for a moment. Well, what are we trying to do? What are you going to observe from the study? You have three studies, right? And the data you observe are actually, what? Always the same. You will get an risk ratio estimate, odds ratio, plus some variance from that study. So, you observe that the amount of information data you have in your hand will stay the same, regardless of which model you are trying to use. And the purpose of doing a meta-analysis. You're trying to use your data you have in your hands and trying to guess where that center of the distribution or where that common effect size is. That's what you are trying to do in meta-analysis. And, we are saying, there are two different ways to get that number, get your meta-analytical results. Either assume the studies are identical, they're the same then, we are going to use a fixed-effect model. If we cannot make that assumption, then we're going to use a random effects model by assuming, well, the studies are slightly different from one another. We're going to assume the true effect sizes are not the same, but there's a distribution. That's what you're doing. You're taking the data you have, you collected from each individual study, and trying to make a best guess where the common effect is. And that guess depends on how different or how similar the studies are. If they are identical, then go ahead, use the fixed effects model. If you can not make that assumption, then you're better off with the random effects model. That leads us to the second session of the random effects model, which we are going to show you how to do it.
