So now, let's talk about Rwgs, ICC1, and ICC2.

We will be talking about agreement and reliability.

When we are running multilevel models,

we needed to see if the people or, yeah,

the people in the team or in the group,

they agree or they reliably report on a particular measures that we want.

So with agreement, we are talking about Rwgs,

which is this within group agreement.

The standard for research is that this agreements should be higher than 0.7.

So, an example of agreement is,

let's say that we have three people in this group and we

all say report one, two, and three.

So yeah, high agreement.

Okay, so we all agree that this construct should be one, two and three.

Reliability is a little bit different.

We are talking about consistency of responses among raters.

So, in this same group,

we could have one person saying one, two,

and three; the other person saying two, three,

and four; and the other person saying three, four, and five.

The variance among their own responses is basically the same,

so they are reliable.

So we can have high agreement and low reliability,

and high reliability and low agreement.

Just an example, in a group of two,

we could have a person with one, two,

and three, and other person with three, four, and five.

They don't agree, but they are reliable.

The dispersion is the same between these two people.

OK. And another way to talk about reliability here is

the proportion of variance explained by group membership.

And we have two ways to calculate reliability.

And we in management,

we tend to report both ICC1 and ICC2.

ICC1 is the reliability of a single assessment

and ICC2 is the reliability of the group means.

I added all the equations here in this slide,

so you can think about these equations and understand them by yourselves.

What I also did was to put or to make available to you a file that calculates the Rwg,

ICC1, and ICC2, and this file is available in the descriptions of this video.

If you want to get that file,

just click in that link that is in the description section.

And now, let's talk about the equations.

So when you are running models,

when you're thinking about multilevel analysis,

you'll have a between-unit model and a within-unit model.

With the between-unit model,

that looks pretty much like a regression equation.

OK. You have the dependent variable,

you have the intercept,

and you have the slope and also a error term.

What happens with the within-unit model is that

your intercept and your slope can also vary within that particular group.

So here, your intercept has a mean plus some deviation.

The same thing happens with your slope,

there is a mean and some deviation.

When you see the graphical representation of that,

I think it will become a little bit more clear to you.

So, we could have models in which the teams

vary in the intercept but they have the same slope.

You'll have teams in which the intercept is the same but the slopes are different.

And you also can have teams that both vary the intercept and they slope.

So now, you have this everything put together and you'll have an equation,

a mixed-level equation, in which you'll have between and within information.

One assumption that's critical is that your mu,

your random error for the lower level,

for the within level,

needs to be independent,

normally distributed and with expected value of zero

and variance tau_squared equal to the variance of mu.

So in this session,

we introduced you to multilevel analysis.

We talked about what multilevel models are,

why they are important,

gave you some graphical representations of multilevel,

and also talked about Rwgs, ICC1, and ICC2.

Multilevel Models Technical Elements

Understanding Your Data: Analytical Tools

Meet the Instructors