0:14

The steps that we go through in probability sampling are the following.

We usually have four steps.

Now, some surveys make this more elaborate, but

these are the four basic ones.

The first thing that we want to do is compute what are called base weights, so

those are inverse selection probabilities, and those do apply to probability samples.

As you will see in a second,

they don't apply in the same way to non-probability samples.

So then we adjust base weights to account for

any units that have got an unknown eligibility.

And we talked about this earlier in the previous course, but

the idea there is there are some units that you may not be able to determine

whether they're eligible or not because you can't contact them, for example.

And we make an adjustment for that.

1:12

Then we adjust for nonresponse.

Some units don't cooperate, so

we try to represent them by making a weighting adjustment.

And then finally, we calibrate to population controls.

Now, this is a flow chart of the steps that we go through to implement those

four steps in the earlier slide.

So we start up here.

The first step is compute base weights in step one here.

So those are going to be the inverse of the selection probabilities.

We've gotta keep track of those.

And then we drop down to this diamond, which is a decision point,

and you ask yourself, do I have units with unknown eligibility?

If I do, then I head down this channel and make an adjustment.

If I don't, then I go down here.

I skip that adjustment for

unknown eligibility step and I go to the next step.

So let's suppose we do have units with unknown eligibility.

What we do is we adjust the weight of the ones whose eligibility is known.

Those are the KNs here.

And as part of the operation, you need to have an audit trail.

So what do we do?

We store the file of unknowns, and

we store a file of ineligibles if I find any of those, and you keep track of those.

That's important because if you have to go backwards and

re-execute any of these steps, you need to have the data available to do that.

And you just don't want to remove the unknowns and

the ineligibles and throw them away.

That's bad procedure.

So we make our adjustment.

We store the file of respondents and

non-respondents here in 2c, and then we get another decision point.

Do we have non-responding units?

If we do, then we head off down this way and make an adjustment for non-response.

If we don't, we get to skip that step and drop down to the next one.

So if we do have non-respondents,

what we're going to do is adjust the weights of the eligible respondents.

And we store the file of non-respondents in 3a here, again because

we're trying to make an audit trail where we can backtrack if we need to.

3:42

Then the output of this step is a stored file of respondents with adjusted

weights for both non-response and

unknown eligibility if you had unknown eligibles.

Then we come to another juncture here.

We've got the possibility of using auxiliary data to adjust for

coverage errors and to improve precision, reduce variances.

So if we do have such data, then we head off down this track.

If we don't, we go down this branch and

store the file of respondents, and we're done.

4:23

If we do have auxiliary data, then we use something called calibration estimation,

which was step four in the previous slide.

And you need external control totals in order to do this.

So a kind of a somewhat subtle point is if your external

controls include units that you would consider to be ineligible,

then what you do is you need to include those INs In your

calibration estimation here, because otherwise,

you'll be adjusting weights to external controls

that are too big for just the eligible units.

So that may or may not happen.

It depends on the source of your external controls.

5:31

Now, non-probability samples are a different story.

You can't do quite as many steps in those cases.

For one thing, you don't have any base weights in the traditional probability

sampling sense because you don't have a probability sample to start with.

So you don't have selection probabilities to invert.

6:17

What you have is a collection of data that you got some way,

in a non-probability way.

So you can't do quite the same sort of nonresponse adjustment.

On the other hand, there are methods out there where you can compute

things called "pseudo-inclusion" probabilities.

It's the sort of thing that's done in observational studies where you didn't

have control over randomizing units into, say, control and treatment.

But you're trying to estimate a quasi-assignment probability for units.

So that's what is possible to do here.

So you can use those "pseudo-inclusion" probabilities and invert those and

get kind of a pseudo-base weight.

The most important step, probably,

in these non-probability samples is to calibrate to population control totals.

The idea here is that you're really trying to make up for

real problems in the coverage of your non-probability sample.

That's one of the big functions of it, and

it also can function that way in a probability sample.