0:14

The main goal is to understand how to check covariant balance after matching.

So we'll assume you've already matched and

now you want to determine whether matching was successful.

So in order to determine whether matching was successful,

one thing we can do is we can check covariate balance.

So remember in a randomized trial,

the covariates would automatically be balanced just due to randomization.

And one of the goals of matching was to create a similar kind

of situation where we'll achieve balance after we match.

So to assess whether balance has been achieved,

we can look at a couple of things.

So one is something known as standardized differences.

And so that has to do with whether the means are similar between the two groups.

And you can do this and you should do this without looking at the outcome.

So at this point,

we're still at these change where we're not looking at our outcome yet.

We carried out matching, and now we want to see whether the matching worked.

And if matching didn't work well, we could go back to the first stage and

think about how to match more effectively.

1:39

Alternatively, as opposed to looking at standardized differences,

you could also carry out hypothesis tests.

So you could test whether the means in two groups are the same after matching.

So for example, you could use two sample t-tests for continuous covariates or

chi-square test for discrete covariates, you could report p-values for those tests.

And the main drawback with that approach is that

the p-values are dependent on sample size.

So if you have a large study, there's very good chance some of the covariants

will be significantly different in the sense of having a small p-value.

But the actual difference in the means might be very small.

In a lot of cases, people have large data sets.

And so we don't necessarily want an assessment of balance that's so

dependent on sample size.

And in particular, because we probably don't care about very small differences,

a p-value might not be the best way to do that.

So typically people prefer to use standardized differences.

2:54

What you essentially do is for each covariant, say a given covariant X,

we'll just take the sample mean in both groups.

So sample mean in the treatment group,

which I'm denoting with this here.

So that's the sample mean of the treatment group.

So you've stratified, you just look at the matched pairs on the treated side.

And you take their average value over their particular covariant that you're

interested in.

You do the same thing with the control, so

you're just taking sample averages, taking the difference.

But then we want to scale that, and so

what we'll do then is we'll just take a pooled standard deviation.

So s squared treatment, that is just the sample variance in

the treatment group for that covariant.

S squared control is just a sample variance in the control arm.

We're just dividing by 2, because we're averaging it.

So we're just taking basically the average variants between two groups.

And then taking the square root

to essentially get a pool of standard deviation.

So smd stands for standardized mean difference.

Basically what we have is a difference in means in standard deviation units.

So in smd of 1 would mean that there's a 1 standard deviation difference in means,

which would be a large difference.

So it's always going to be relative to the standard deviation.

And again, the reason for that is because if we change scales,

we don't want the standardized mean difference to change.

So if you go from recording age in years to age in days,

the standardized mean difference will not change and that's how we would want it.

So we shouldn't be sensitive to changes of scale.

4:40

And so a key feature of the standardized difference is that it's not

directly dependent on sample size.

If you look at the formula here, you'll see that there's no sample size in there.

So there's no n, if you think of n as a sample size, for example,

it doesn't appear in there.

5:01

And often people report the absolute value of the standardized difference.

So you'll notice here, this could either be a positive or negative value,

because in the numerator, we have a difference in means.

And that could either be positive or negative,

so a lot of times people actually take the absolute value of the whole thing.

5:29

And then there's some rules of thumb that are commonly used.

So a standardized mean difference of less than 0.1 in absolute value

would typically indicate adequate balance.

So if you have a randomized trial for example, and

you calculate a standardized difference between groups on each covariant,

typically they'll all be less than 0.1.

They'll use between 0.1 and 0.2 are considered not balanced necessarily,

but small enough for maybe you're not very worried about it.

So even if you do a good job of matching a lot of times, there will be a couple of

co-variance that are between 0.1 and 0.2 in terms of standardized difference.

And usually people don't worry a lot about that, but

value is greater than 0.2 are considered serious imbalance.

6:16

So now we will look at what a table 1 would look like.

Table 1 is kind of a generic term because it's a kind of table that often

appears first in a manuscript.

So it's just a generic name for this kind of a table.

So next, I'll show an example of a table 1.

So table 1 is just a generic term for

the kind of table that often appears in a manuscript as the first table.

But it's just summary statistics of the main variables involved in the analysis.

And if you're carrying out unmatched analysis or if you're comparing

treatments, typically you'll stratify the table on the treatment groups.

So here's an example of data from the right heartncatheterization study,

where right heart catheterization is the treatment or RHC.

7:27

Then the first variable that we see is age here.

And so this is showing that the mean and

standard deviation of age in the two groups.

So this is unmatched to just stratify and treatment group and

then calculate the mean and standard deviation for each of those.

So we see that the average age in the two groups is roughly 61.

They're very similar and the standardized mean difference is 0.06.

The second variable is sex so percent male and

then we have a few other variables that have to do with medical diagnosis.

And I'm only showing a few variables that's just for illustration and

practice that would typically be dozens of variables that you would include in our

table 1.

So this is just for illustration, but you can see here for this neuro covariant.

The prevalent is 16.2% in the control group versus 5.4% in

the treatment group with standardized mean difference of 0.35.

And we call that, we said that anything greater that 0.2 is

something you would probably be concerned about.

And so this is suggesting that there is imbalance there.

8:39

And next, if you're carrying on a matched analysis, you would typically include in

a table the same information but for the matched groups.

So now we've matched and you'll see that now we have exactly the same

number of people in both groups, so we did paired matching.

So for everyone in the control group,

we found somebody in the treatment group that was matched well in the covariates.

9:05

And now we see that the standardized mean differences here are all much smaller.

And in fact, they're all less than 0.1 in absolute value.

And in particular, we could highlight the narrow category where

previously was 16.2% versus 5.4%.

Now it's 5.3% versus 5.7%, and

so the second half of this table here, the matched case.

These numbers look much more like what you would see in a randomized trial,

where you basically have the same number of people in each group.

And you have a lot of balance between the two groups.

So this is what we hope will happen after matching,

is that the table would have looked had we randomized.

9:50

You can also plot the standardized mean differences and

this is especially useful if you have many covariants.

So here, I just included these five covariants, but

typically again you might have dozens of them.

And a plot like this, can really show overall how well matching did.

So the red curve here, the red lines,

10:12

those are from the unmatched population.

It's displaying standardized mean difference for each variable.

And this is just plotting the numbers I had on the previous slide.

And we have a solid line at 0.1 here,

because that's sort of a default cut off that people often use for

what's considered a good match versus not.

And you'll see that the red line, a lot of the variables are to

the right of the 0.1 cut off, so they're on this side of it.

Whereas for the blue, they're all to the left, right?

And the blue here, the blue line is the matched.

So what this is showing is that matching was able to create

much better balance on these covariants.

So this kind of graph is very useful if you have dozens of covariants, and

you can see if overall matching created balance between the groups.