A practical and example filled tour of simple and multiple regression techniques (linear, logistic, and Cox PH) for estimation, adjustment and prediction.

Loading...

From the course by Johns Hopkins University

Statistical Reasoning for Public Health 2: Regression Methods

76 ratings

A practical and example filled tour of simple and multiple regression techniques (linear, logistic, and Cox PH) for estimation, adjustment and prediction.

From the lesson

Module 3A: Multiple Regression Methods

This module extends linear and logistic methods to allow for the inclusion of multiple predictors in a single regression model.

- John McGready, PhD, MSAssociate Scientist, Biostatistics

Bloomberg School of Public Health

Greetings, and welcome.

Â It's Lecture Set 7.

Â In this set of lectures,

Â we'll do a parallel treatment of logistic regression, like we did for

Â linear regression, for estimation, adjustment, and basic prediction.

Â So, in the first section,

Â we'll look at some examples of multiple logistic regression.

Â We'll also get back to doing the same thing in Lecture Set C where we'll look at

Â examples from the published literature.

Â So hopefully, after viewing this section, you'll have some sense of how to interpret

Â the estimates from multiple logistic regression models in a substantive or

Â scientific context.

Â And compare the results from simple and

Â multiple logistic regression models to this past potential confounding.

Â So let's first look at the data from a random sample of 192 Nepali children

Â between a year and three years old, 12 to 36 months old.

Â We already looked at the relationship between breast feeding and

Â age in this group.

Â We looked initially at the relationship between breast feeding sex,

Â in the group that included children between 0 and 12 months.

Â But we're going to restrict our analysis now to this group that's a year or older,

Â up to three years.

Â So in this sample,

Â 70% of the children were being breast fed at the time of the study.

Â In the samples 48% female and

Â we're going to have a sex variable that's coded as 1 for males and 0 for females.

Â So here's the result, the unadjusted association, in the sample.

Â But, that says the log odds of being breast fed is a function of

Â this intercept 1.12 plus 0.02.

Â Times the sex of the child.

Â Where 1 is for males and 0 is for females.

Â So this equation is only estimating 2 log odds.

Â 1 for females which is the reference group and 1 for males.

Â This slope estimate of 0.02 is the log odds ratio of being breast fed for

Â males compared to females.

Â If we exponentiate this, we'll get nons ratio estimate of approximately 1.02.

Â So, in the sample, males have a slightly higher odds.

Â We could get a confidence level for this by taking the estimate plus or

Â minus 2 times the standard error which I haven't given you in this slide.

Â But the idea is exactly the same as we were doing before.

Â And then this intercept of 0.83 estimates the log odds of being breast fed for

Â female children in the sample.

Â And we could translate that, exponentiate it to get the odds for

Â female children, and estimate the probability or proportion from that.

Â Now, let's bring in age of the child.

Â We saw before a negative association, not surprisingly between the age of

Â the child and breastfeeding status in this age group.

Â Let's put sex and age together and see what the results look like.

Â So here is the result which is the regression equation that includes sex as

Â a predictor.

Â 1 for males and 0 for females and age and months as a continuous predictor.

Â When we'd all ready visually assessed and that by a low s plot and

Â solve it at least, and

Â the other adjusted case that made sense to model it as a linear function.

Â So.

Â The estimated association.

Â The log odds ratio from males to females is 0.27.

Â If we were to put a confidence interval in that, and we could do that by

Â taking the estimate plus or minus 2 times its estimated standard error, but

Â I'll just jump to the, we're so familiar with doing that now.

Â I'll just jump to the results here.

Â Even though this is a multiple regression model, but

Â the approach is exactly the same for

Â confidence intervals as we've seen for slopes from simple regression models.

Â So, the slope or the log odds ratio of males to females, it was positive.

Â But, after accounting for sampling variability, the confidence interval

Â includes 0, and the resulting pvalue for testing the null that the slope is 0.

Â Where that the resulting adjusted odds ratio is 1, is 0.48.

Â So after accounting for.

Â Age the child,

Â we don't see a statistically significant association between the.

Â Log odds of being breast fed and the sex of the child.

Â However, the slope for.

Â Age is statistically significant.

Â All possibilities in the confidence interval are negative and the pvalue for

Â testing the null, that there's no association between breast feeding and

Â age after accounting for sex is small.

Â So let's parse these estimates a little bit further,

Â make sure we detail what they mean.

Â So, the slope estimate for sex is beta 1 hat equals 0.27.

Â And it's still an estimated log odds ratio of breast feeding for

Â male children to female children, but

Â it's removed any difference in the age distributions between those.

Â We've adjusted for age.

Â So, this number compares male children to female children.

Â Of the same age.

Â And this is called the age adjusted association between breast feeding

Â and sex.

Â So, the log odds ratio was 0.27.

Â We know to get the odds ratio we would just exponentiate that.

Â And that turns out to be 1.3.

Â So, male children in the sample have 30% greater odds of

Â being breast fed than female children in the sample of the same age.

Â However, when we hold this up, for when we account for

Â the uncertainty in the estimate and

Â see if it holds at the population level, what we already saw in the log odd scale.

Â The confidence interval included 0.

Â We exponentiate those endpoints to get a confidence interval for the.

Â Population level association on the odds ratio scale.

Â The resulting confidence interval goes from 0.61 to 2.82 and

Â includes the null value of 1.

Â So after accounting for sampling vali,

Â variability, sex is not associated with breast feeding after accounting for age.

Â And it wasn't associated even when we ignored age in the simple model.

Â We'll look at the estimate for age, the slope is negative 0.24.

Â This is still an estimated log odds ratio of breast feeding for

Â children who differ by one month in age, older to younger.

Â But now we've removed any differences in the sex distributions across

Â the age groups.

Â So this compares children who are of the same sex, males to males who differ by

Â one month in age or females to females who differ by one month in age.

Â And this is called sex adjusted association between

Â breast feeding and age.

Â So the resulting odds ratio estimate here if we exponentiate this is 0.79.

Â Suggests that a one month difference in age is associated with

Â a 21% reduction in the odds beaten, being breast fed, for

Â older compared to younger among children of the same sex.

Â And then 95% confidence level for the population level, sex adjusted odds ratio.

Â If we just exponentiate those endpoints of the confidence interval for

Â the slope, it goes from 0.73 to 0.84.

Â After counting for sample availability,

Â there's clear evidence of a population level association.

Â And while we estimate a 21% reduction,

Â this could be anywhere on, up to a 27% reduction.

Â Or as quote, unquote, small as a 16% reduction in the odds per month of age.

Â So, how could we present the findings from our simple and

Â adjust, adjusted models together?

Â Well and like seen with linear aggression in research articles, frequently a single

Â table of unadjusted and adjusted associations will be presented.

Â So we, if we were just putting the results here, we could do something like this.

Â We could put the unadjusted column, which would and

Â look at the way I've handled sex, here I've said male and female.

Â And for the unadjusted odds ratio for

Â females, I put a 1, that's a way of indicating that it's the reference group.

Â There's no confidence limits, et cetera.

Â This is what the other levels are being compared to.

Â Of course, there's only one other level of sex, which is male.

Â In this pr, the unadjusted.

Â Odds ratio and confidence interval.

Â And then, this does the same thing.

Â This does the same thing for age, the unadjusted, you know.

Â And then, in this column here it shows the results from

Â the model that includes both sex and age.

Â Well, let's just take a look at this for a minute.

Â Certainly the relationship, the estimated relationship between sex and

Â age changed from an odds ratio of 1.02 to 1.3 for

Â males compared to females, after adjusting for age.

Â So it appears that within the sample there might have been a slight discordance in

Â the age distributions between males and females that was.

Â May gain some of the associations, but

Â of course, this association is not statistically significant, and

Â the confidence intervals overlap a fair amount.

Â So, I would say that there, in general qualitatively, there was no

Â overall confounding of the sex relationship after we accounted for age.

Â Similarly, if we look at the age association,

Â it's identical to what it was when we ignored sex.

Â Of the same confidence intervals.

Â So, it's pretty clear that the,

Â regional association we saw between breast feeding and age of the child

Â was not attributable at all to any sex differences between the age group.

Â One more thing that you'll sometimes see in papers, and

Â this looks crazy when you exponentiate that intercept of 7.2, it is on the order.

Â Of a 1,333 and that's the baseline odds.

Â This is the, the group odds when age is zero, and

Â we don't have any newborns in this sample.

Â Remember, we started at 12 months and

Â when we are looking at females, the reference group.

Â So, this expresses sort of the starting odds, wherever you start.

Â When we make these comparisons.

Â And even though we don't describe a single group,

Â it's worth noting that if this is the starting odds,

Â then the estimated probability of being breast fed, that we're working off of,

Â as a, a reference on the log odd scale is very close to.

Â 1.

Â Remember, we just take the odds over the odds,

Â plus 1 to get the estimated probability.

Â So even though there isn't a group in our sample that's newborn and female.

Â Because we don't start with children till they're 12 months old,

Â this tells us that the st,

Â the general ideas were starting with a very high point and coming down from that.

Â And remember the over all proportion of people being breast fed in

Â this sample was on the order of 70%.

Â So by the time we actually get into estimates with our x

Â values that are relevant to our data set,

Â that will reduce it from this high starting point of.

Â A really large odds to begin with.

Â So, there's some other additional predictors in

Â these data that we could look at.

Â One is maternal parity.

Â And it has, actually I put it into five groups for these data.

Â About 17% of the sample, the child we're looking at now, is their first child.

Â They had no previous children prior to this one.

Â Another 16% had one previous child.

Â Another 14% had two previous children, before the one we're looking at and

Â an, analyzing the breast feeding status of.

Â Another 15% had three previous children.

Â And over a third of the sample, 38% had greater than or

Â equal to four previous children.

Â Also of interest, and it's, could be very well related to the parity as well,

Â although it doesn't necessary have to be, is the maternal age.

Â In the sample the age, the mean age of the mothers for

Â these children, is 27.7 years with a range of 17 to 43 years.

Â So, I'm just actually going to present the results from several models

Â side-by-side to have us take a look at what's going on.

Â So, here is the unadjusted column.

Â This shows the associations between sex and age we already talked about.

Â Let's look at what's going on with maternal parity.

Â It looks like, with regards to the reference,

Â which is no previous children, children whose mothers had.

Â At least one prior child have lower odds of being breast-fed

Â than those whose mothers had previous children.

Â This doesn't take into account any other characteristics,

Â as it's the unadjusted association.

Â But you'll see these confidence intervals all are very wide and

Â include the null value of one.

Â And in fact if we test.

Â The overall construct that there's any differences in the odds

Â of breast feeding among children for any comparison of these parity groups.

Â And the nice thing about this overall pvalue is it also tests behind

Â the scenes the differences between this group.

Â It, it doesn't just task the comparisons to the reference, which is all we,

Â the, all that we see with the odds ratios.

Â And so, the null is that there's no association between breast feeding

Â and the parity.

Â There's no differences in the odds across any of the parity groups, and,

Â and the unadjusted level, we would fail to reject that.

Â We didn't find anything here and mother's age.

Â As an increase in mother's age is associated with a slight decrease in

Â the odds of being breast fed, but it's not statistically significant.

Â So, if we go forth to the adjusted associations,

Â I'm just going to let you look at these a little bit, but

Â you'll see that very little changes about the story with age.

Â We certainly didn't see a change at all when we adjusted for sex.

Â But when we adjusted additionally this model here adjusts for maternal parity.

Â And this final model here adjusts includes male,

Â sex, maternal parity, and maternal age.

Â And you can see that the the age association is robust.

Â It's stays pretty much the same regardless of the other things in the model.

Â Similarly, the story with sex remains in

Â that there's no statistically significant association or

Â difference between the odds of being breast fed from males and females whether.

Â We don't consider these other factors in the unadjusted sense, or

Â we consider an adjust for age.

Â Or add in maternal parity or add in on top of that mother's age,

Â the story is pretty much the same across these adjusted estimates.

Â Similarly, if we look at maternal parity.

Â I won't walk you through these, but when we include it after including sex and

Â age, it's, doesn't become a statistically significant predictor.

Â And in fact, the odds ratios look slightly different,

Â the estimates, than they did for some of the groups.

Â But it doesn't appear that qualitatively there's much of a difference here.

Â And certainly, statistically speaking, there's not.

Â And, in that model includes everything, sex, age of the child,

Â maternal parity, and maternal age.

Â Maternal parity is not significant, not a significant predictor either.

Â Nor is maternal age.

Â It doesn't become so after adjusting for these other things.

Â So, on the whole I think the big story there is

Â that the only predictor that holds up and consistently holds up at all,

Â not just consistently so, is the age of the child.

Â Among these candidates, there don't appear to be differences related to sex,

Â maternal parity, or mothers age.

Â With or without considering each other in adjusted models.

Â Let's just focus on this diagramming this Model 4 for a minute.

Â This just comes from multiple linear regression with a bunch of xs.

Â And so if you actually looked at the model.

Â Here is the I'm just going to write the, the counterpart to each of these.

Â This is the baseline odds.

Â This is the exponentiated intercept.

Â So on the log scale, the log odd scale.

Â The original regression scale.

Â This intercept would be the log, natural log of 7,071, which is 9.1.

Â And if we actually took the logs of each of these slopes.

Â I'll just put them in.

Â So that's for, sorry that's for parity.

Â For maternal age, it's negative 0.017.

Â And for age, it's negative 0.26 and then for sex.

Â It's 0.21, the only positive coefficient we have here.

Â So, this is just it on a log scale, and so the equation that gave us these estimates

Â was the log odds of being breastfed equals 9.1 plus neg,

Â negative 0.017 times mother's age in years.

Â Et cetera, et cetera.

Â And these estimates come from the exponeniated coefficients.

Â And all of these confidence intervals were first done by the computer on

Â the log ratio scale by taking the estimate for each and adding and

Â subtracting 2 estimated standard errors given by the computer.

Â And then exponentiating those end points to get the confidence interval.

Â So, lets look at our data from 2009 to 10 NHANES to give another example of

Â multiple logistic regression and comparing the results from several models.

Â This is, we initially looked at HDL cholesterol levels and whether is

Â predicted obesity in the population from which the sample was taken.

Â 6400 US residents,16 to 80 years old,

Â so that's a population of 16 to 80 year old US residents.

Â And we saw that the HDL levels, the average was 52.4 milligrams per deciliter.

Â There was a substantial variability in the sample and

Â 15% of the samples obese by BMI.

Â So, some other potential predictors we might want to look at include sex,

Â the age of the years, the age in years of the person, and their marital status.

Â We're trying to get a demographic and

Â physiological overview of predictors of obesity using these data.

Â So, just some things to consider, just to let you know.

Â In this sample 49% of the sample was female, 51% was male.

Â The average age in the sample was 46.3 years ranging from 16

Â to 80 as we talked about before.

Â And, there were actually six different categories of marital status.

Â Married in which a little over half the sample identified as.

Â Another 9% were widowed.

Â Another 11% had been divorced.

Â 3% were separated.

Â 18% had never been married.

Â And the remaining 7% classified themselves as in

Â a relationship where they were cohabiting or living with a partner.

Â So, I just want to show you some things I looked at before moving forward with this.

Â I wanted to get a sense of what the obesity,

Â age relationship looked like because that was measured on a continuum, in years.

Â And here are the results from a lowess plot that

Â shows the unadjusted association.

Â And you can see if, if it doesn't appear that this is well described by a line.

Â In fact if I fit, if I fit a line in an assumed linear relationship,

Â the line I get would, the best fitting line for this picture would probably miss

Â the story because it would probably be close to flat.

Â So, there's a couple of things we'll talk,

Â later lecture, about how we might handle this literally, but

Â there's, there's procedures you can use to fit a changing slope.

Â A slope that changes at different points over the relationship to allow for

Â the association to be non-linear.

Â But another catch-all method to kind of deal with this is just to

Â create groups of age and model that as a categorical predictor.

Â So that's what I chose to do, I broke age into quartiles for these data.

Â First group was from the minimum to the 25th percentile,

Â second group from the 25th percentile to the 50th, etc.

Â So, let's look at the results from some regression models actually putting this

Â all together.

Â So I'm first going to focus on the unadjusted associations with

Â each of these things.

Â So let's just, this is the result we got before with HDL.

Â The odds ratio of being obese for

Â two groups who differ by one milligram per deciliter.

Â Is, I'm going to take it to three decimal places because otherwise

Â the confidence inter, endpoints would round to the estimate, and

Â it would have looked a little confusing.

Â So, the odds ratio here was 0.967, so

Â a relative decrease in the odds of being obese by 3.3%.

Â Per one milligram deciliter difference in cholesterol level.

Â And this, at the unadjusted level, as we saw before, was statistically significant

Â in the confidence interval for the ratio does not include one.

Â If we compared males to females, no other factors considered, males had a higher,

Â odds by 75% of being obese compared to females in the sample.

Â And if we then accounted for sampling variability, there was,

Â was statistically significant.

Â Each category was, was in,

Â on the whole associated as this, the pvalue testing the null.

Â That all the associations, all the category comparisons are,

Â are equal to 0 in the log scale, where all the odds ratios are equal to 1.

Â In other words the null of no association between obesity and

Â age when age is modeled as four categories.

Â That would be rejected, and we can see that actually for the most part.

Â Increasing age is associated with increased risk of being obese,

Â relative to the reference group of being less than 30 years old.

Â But it's pretty similar comparison for the second quartile,

Â the group 30 to 46 years old and the group 46 to 62 years old.

Â 1.79 and 1.82, respectively with very similar confidence intervals.

Â And then the estimate shifts down a bit.

Â It's still greater than one, statistically significant for

Â the group, that is greater than or equal to 62 years.

Â But is a smaller estimated odds ratio when

Â compared to the same reference as the other two.

Â If we look at marital status, interestingly enough it doesn't appear

Â that in the adju, unadjusted sense, there is any association between marital status.

Â The null is not rejected and the pvalue is 0.69.

Â But you can see that the widowed, divorced, and

Â separated categories all have higher estimated odds, but the re,

Â differences not statistically significant when compared to the married group.

Â Whereas the never married has almost equivalent odds,

Â being obese compared to the married group.

Â And the living together group has slightly lower odds in the sample.

Â But after counting for

Â sampling variability, it's not statistically significant.

Â So let's look at Model 2.

Â Model 2 is the multiple model that includes HDL, sex, and age as predictors.

Â And let's see what the results are,

Â compare the adjusted associations from this model to the previous unadjusted.

Â So, we can see that the association with HDL is still

Â such that increased HDL is associated with decreased odds of being obese.

Â It's similar magnitude, to the unadjusted estimate and

Â still statistically significant.

Â So, it doesn't look like that association we initially saw was greatly explained

Â by sex or age differences in the HDL group, so very little confounding here.

Â Interestingly enough though, the sex comparison increases and

Â the confidence interval shifts up to what it was when we ignored HDL and age.

Â And it looks like if we were to compare males to females of the same age and

Â HDL levels, the males would have 2.6 times the odds of being obese.

Â And that's statistically significant, and the confidence interval is different.

Â It shifts up compared to what it was in the unadjusted.

Â So, it looks like some of the male female relationship was being dampened.

Â Because of behind the scenes relationships between obesity,

Â sex, HDL, and potentially age.

Â If you look at the age association,

Â it's still statistically significant on the whole, and

Â there is some movement in the estimates compared to the unadjusted counterparts.

Â But qualitatively speaking the same ordering of associations, and

Â they're all statistically significant.

Â So, really the biggest story about confounding was with the,

Â with the sex association.

Â It appeared to shift upwards and

Â differ statistically after adjustment when compared with the unadjusted.

Â In Model 3, just for, just to look at the sensitivity to bringing in marital status,

Â I, I added that to the model that we just looked at.

Â It had very little impact on the first two, relative to

Â what they were when we adjusted only, when we only included HDL, sex, and age.

Â The age category comparison attenuated a bit after counting additionally for

Â marital status.

Â But qualitatively the same ordering exists.

Â And again, the results are statistically significant indicating that

Â generally older age,

Â beyond 30, is generally associated with increased odds relative less than 30.

Â And if you look carefully at the marital status of the estimates and

Â the confidence levels, the significance

Â remained the overall pvalue for testing this association was on the order of 0.5.

Â That test, they know that marital status is non-associated with

Â obesity after accounting for HDL, sex, and age.

Â And if you look at the estimates here, some of them change a little bit.

Â The confidence intervals are all very wide.

Â But I think the general story is that.

Â These models show that HDL, sex, and age are significant predictors of obesity.

Â And generally with the exception of sex,

Â did not impact each other as association with the outcome of obesity.

Â And after additionally accounting for

Â marital status, marital status doesn't add anything statistically.

Â And doesn't change our results for the other three.

Â So again, and we could look at this model, for example, the one with age.

Â And we could figure out what the coefficients were,

Â not that we want to work backwards.

Â But this does come from a logistic regression model that

Â started off of the form, a log odds of obesity.

Â Equals some intercept estimate.

Â And then we'd have our three slopes for

Â the three indicators of age group because this is being treated as categorical.

Â This would be our age, for example.

Â And you could, and then we'd have something for being male, versus female.

Â And then we'd have something for.

Â And it really doesn't matter what you name these coefficients and

Â xs, just know that there are three predictors.

Â HDL, sex, and age.

Â And, then if we wanted to get the re, corresponding coefficients and

Â the intercept, we could take the log of the estimates given here.

Â Not that we want to do this backwards,

Â but generally, this is where these results came from.

Â And the confidence intervals came from behind the scenes.

Â The computer estimated the regression slopes, and then a standard error, and

Â added and subtracted 2 standard errors, these estimates, business as usual.

Â And then, exponentiated those to get the end points on the odds ratio scale.

Â So, in summary logistic regression is a tool to allow us to look at

Â binary outcomes as a function of multiple predictors at once.

Â And we've already defined, in Lecture 2, a simple linear regression.

Â And we talked about how our slopes are interpretable as log odds ratios, to how

Â they can be ext, extended, exponentiated to become odds ratio estimates.

Â How we can create confidence intervals for the slope and exponentiate those results,

Â to get confidence intervals for the odds ratio of interest.

Â But we can just extend that with multiple regression,

Â we can look at the impact of potentially multiple predictors on the binary outcome.

Â We can get adjustments of the adjusted association between each predictor and

Â the outcome adjusted for the other variables in the model.

Â And then, we compare these adjusted estimates and

Â their confidence intervals to the unadjusted estimates to get some sense of

Â the degree of confounding if there is any.

Â And we can also decide which of these predictors add information about

Â the outcome, versus which do not by looking at the statistical significance.

Â So, it's just a very nice logical extension of what we did with

Â simple logistic regression and the ideas in terms of the interpretation parallel.

Â Exactly what we do with multiple linear regression.

Â This is really just another form of the same thing.

Â It just so happens that the scale on which the estimates are is different.

Â So, in the next section we'll talk a little bit about,

Â the the basics of using the multiple regression models to compare the odds for

Â groups who differ by more than one predictor.

Â And to estimate probabilities proportions of

Â persons having the outcome given their x values.

Â And then in the third section, we'll look at some examples of logistic regression

Â from the published, public health, and medical literature.

Â Coursera provides universal access to the worldâ€™s best education,
partnering with top universities and organizations to offer courses online.