Hi, this video is on assessing balance. The main goal is to understand how to check covariant balance after matching. So we'll assume you've already matched and now you want to determine whether matching was successful. So in order to determine whether matching was successful, one thing we can do is we can check covariate balance. So remember in a randomized trial, the covariates would automatically be balanced just due to randomization. And one of the goals of matching was to create a similar kind of situation where we'll achieve balance after we match. So to assess whether balance has been achieved, we can look at a couple of things. So one is something known as standardized differences. And so that has to do with whether the means are similar between the two groups. And you can do this and you should do this without looking at the outcome. So at this point, we're still at these change where we're not looking at our outcome yet. We carried out matching, and now we want to see whether the matching worked. And if matching didn't work well, we could go back to the first stage and think about how to match more effectively. And so in a lot of cases, what we'll do is we'll start by looking at a table 1 essentially. So a table that looks at pre-matching versus post-matching balance. Alternatively, as opposed to looking at standardized differences, you could also carry out hypothesis tests. So you could test whether the means in two groups are the same after matching. So for example, you could use two sample t-tests for continuous covariates or chi-square test for discrete covariates, you could report p-values for those tests. And the main drawback with that approach is that the p-values are dependent on sample size. So if you have a large study, there's very good chance some of the covariants will be significantly different in the sense of having a small p-value. But the actual difference in the means might be very small. In a lot of cases, people have large data sets. And so we don't necessarily want an assessment of balance that's so dependent on sample size. And in particular, because we probably don't care about very small differences, a p-value might not be the best way to do that. So typically people prefer to use standardized differences. So here's what a standardized difference is. What you essentially do is for each covariant, say a given covariant X, we'll just take the sample mean in both groups. So sample mean in the treatment group, which I'm denoting with this here. So that's the sample mean of the treatment group. So you've stratified, you just look at the matched pairs on the treated side. And you take their average value over their particular covariant that you're interested in. You do the same thing with the control, so you're just taking sample averages, taking the difference. But then we want to scale that, and so what we'll do then is we'll just take a pooled standard deviation. So s squared treatment, that is just the sample variance in the treatment group for that covariant. S squared control is just a sample variance in the control arm. We're just dividing by 2, because we're averaging it. So we're just taking basically the average variants between two groups. And then taking the square root to essentially get a pool of standard deviation. So smd stands for standardized mean difference. Basically what we have is a difference in means in standard deviation units. So in smd of 1 would mean that there's a 1 standard deviation difference in means, which would be a large difference. So it's always going to be relative to the standard deviation. And again, the reason for that is because if we change scales, we don't want the standardized mean difference to change. So if you go from recording age in years to age in days, the standardized mean difference will not change and that's how we would want it. So we shouldn't be sensitive to changes of scale. And so a key feature of the standardized difference is that it's not directly dependent on sample size. If you look at the formula here, you'll see that there's no sample size in there. So there's no n, if you think of n as a sample size, for example, it doesn't appear in there. So it's not dependent on sample size in the way a p-value would be. And often people report the absolute value of the standardized difference. So you'll notice here, this could either be a positive or negative value, because in the numerator, we have a difference in means. And that could either be positive or negative, so a lot of times people actually take the absolute value of the whole thing. And then we would do this for every variable that we matched on. So one at a time, we'll go through and we'll calculate a standardized mean difference. And then there's some rules of thumb that are commonly used. So a standardized mean difference of less than 0.1 in absolute value would typically indicate adequate balance. So if you have a randomized trial for example, and you calculate a standardized difference between groups on each covariant, typically they'll all be less than 0.1. They'll use between 0.1 and 0.2 are considered not balanced necessarily, but small enough for maybe you're not very worried about it. So even if you do a good job of matching a lot of times, there will be a couple of co-variance that are between 0.1 and 0.2 in terms of standardized difference. And usually people don't worry a lot about that, but value is greater than 0.2 are considered serious imbalance. So now we will look at what a table 1 would look like. Table 1 is kind of a generic term because it's a kind of table that often appears first in a manuscript. So it's just a generic name for this kind of a table. So next, I'll show an example of a table 1. So table 1 is just a generic term for the kind of table that often appears in a manuscript as the first table. But it's just summary statistics of the main variables involved in the analysis. And if you're carrying out unmatched analysis or if you're comparing treatments, typically you'll stratify the table on the treatment groups. So here's an example of data from the right heartncatheterization study, where right heart catheterization is the treatment or RHC. So here we'll see, RHC is the treatment group, and no RHC is the control group. And you'll see that in this table, there's a few thousand people in each group. So we'd begin with just the sample size, and it says how many subjects are in each group. Then the first variable that we see is age here. And so this is showing that the mean and standard deviation of age in the two groups. So this is unmatched to just stratify and treatment group and then calculate the mean and standard deviation for each of those. So we see that the average age in the two groups is roughly 61. They're very similar and the standardized mean difference is 0.06. The second variable is sex so percent male and then we have a few other variables that have to do with medical diagnosis. And I'm only showing a few variables that's just for illustration and practice that would typically be dozens of variables that you would include in our table 1. So this is just for illustration, but you can see here for this neuro covariant. The prevalent is 16.2% in the control group versus 5.4% in the treatment group with standardized mean difference of 0.35. And we call that, we said that anything greater that 0.2 is something you would probably be concerned about. And so this is suggesting that there is imbalance there. And next, if you're carrying on a matched analysis, you would typically include in a table the same information but for the matched groups. So now we've matched and you'll see that now we have exactly the same number of people in both groups, so we did paired matching. So for everyone in the control group, we found somebody in the treatment group that was matched well in the covariates. And now we see that the standardized mean differences here are all much smaller. And in fact, they're all less than 0.1 in absolute value. And in particular, we could highlight the narrow category where previously was 16.2% versus 5.4%. Now it's 5.3% versus 5.7%, and so the second half of this table here, the matched case. These numbers look much more like what you would see in a randomized trial, where you basically have the same number of people in each group. And you have a lot of balance between the two groups. So this is what we hope will happen after matching, is that the table would have looked had we randomized. You can also plot the standardized mean differences and this is especially useful if you have many covariants. So here, I just included these five covariants, but typically again you might have dozens of them. And a plot like this, can really show overall how well matching did. So the red curve here, the red lines, those are from the unmatched population. It's displaying standardized mean difference for each variable. And this is just plotting the numbers I had on the previous slide. And we have a solid line at 0.1 here, because that's sort of a default cut off that people often use for what's considered a good match versus not. And you'll see that the red line, a lot of the variables are to the right of the 0.1 cut off, so they're on this side of it. Whereas for the blue, they're all to the left, right? And the blue here, the blue line is the matched. So what this is showing is that matching was able to create much better balance on these covariants. So this kind of graph is very useful if you have dozens of covariants, and you can see if overall matching created balance between the groups.
