0:14

The steps that we go through in probability sampling are the following.

Â We usually have four steps.

Â Now, some surveys make this more elaborate, but

Â these are the four basic ones.

Â The first thing that we want to do is compute what are called base weights, so

Â those are inverse selection probabilities, and those do apply to probability samples.

Â As you will see in a second,

Â they don't apply in the same way to non-probability samples.

Â So then we adjust base weights to account for

Â any units that have got an unknown eligibility.

Â And we talked about this earlier in the previous course, but

Â the idea there is there are some units that you may not be able to determine

Â whether they're eligible or not because you can't contact them, for example.

Â And we make an adjustment for that.

Â 1:12

Then we adjust for nonresponse.

Â Some units don't cooperate, so

Â we try to represent them by making a weighting adjustment.

Â And then finally, we calibrate to population controls.

Â Now, this is a flow chart of the steps that we go through to implement those

Â four steps in the earlier slide.

Â So we start up here.

Â The first step is compute base weights in step one here.

Â So those are going to be the inverse of the selection probabilities.

Â We've gotta keep track of those.

Â And then we drop down to this diamond, which is a decision point,

Â and you ask yourself, do I have units with unknown eligibility?

Â If I do, then I head down this channel and make an adjustment.

Â If I don't, then I go down here.

Â I skip that adjustment for

Â unknown eligibility step and I go to the next step.

Â So let's suppose we do have units with unknown eligibility.

Â What we do is we adjust the weight of the ones whose eligibility is known.

Â Those are the KNs here.

Â And as part of the operation, you need to have an audit trail.

Â So what do we do?

Â We store the file of unknowns, and

Â we store a file of ineligibles if I find any of those, and you keep track of those.

Â That's important because if you have to go backwards and

Â re-execute any of these steps, you need to have the data available to do that.

Â And you just don't want to remove the unknowns and

Â the ineligibles and throw them away.

Â That's bad procedure.

Â So we make our adjustment.

Â We store the file of respondents and

Â non-respondents here in 2c, and then we get another decision point.

Â Do we have non-responding units?

Â If we do, then we head off down this way and make an adjustment for non-response.

Â If we don't, we get to skip that step and drop down to the next one.

Â So if we do have non-respondents,

Â what we're going to do is adjust the weights of the eligible respondents.

Â And we store the file of non-respondents in 3a here, again because

Â we're trying to make an audit trail where we can backtrack if we need to.

Â 3:42

Then the output of this step is a stored file of respondents with adjusted

Â weights for both non-response and

Â unknown eligibility if you had unknown eligibles.

Â Then we come to another juncture here.

Â We've got the possibility of using auxiliary data to adjust for

Â coverage errors and to improve precision, reduce variances.

Â So if we do have such data, then we head off down this track.

Â If we don't, we go down this branch and

Â store the file of respondents, and we're done.

Â 4:23

If we do have auxiliary data, then we use something called calibration estimation,

Â which was step four in the previous slide.

Â And you need external control totals in order to do this.

Â So a kind of a somewhat subtle point is if your external

Â controls include units that you would consider to be ineligible,

Â then what you do is you need to include those INs In your

Â calibration estimation here, because otherwise,

Â you'll be adjusting weights to external controls

Â that are too big for just the eligible units.

Â So that may or may not happen.

Â It depends on the source of your external controls.

Â 5:31

Now, non-probability samples are a different story.

Â You can't do quite as many steps in those cases.

Â For one thing, you don't have any base weights in the traditional probability

Â sampling sense because you don't have a probability sample to start with.

Â So you don't have selection probabilities to invert.

Â 6:17

What you have is a collection of data that you got some way,

Â in a non-probability way.

Â So you can't do quite the same sort of nonresponse adjustment.

Â On the other hand, there are methods out there where you can compute

Â things called "pseudo-inclusion" probabilities.

Â It's the sort of thing that's done in observational studies where you didn't

Â have control over randomizing units into, say, control and treatment.

Â But you're trying to estimate a quasi-assignment probability for units.

Â So that's what is possible to do here.

Â So you can use those "pseudo-inclusion" probabilities and invert those and

Â get kind of a pseudo-base weight.

Â The most important step, probably,

Â in these non-probability samples is to calibrate to population control totals.

Â The idea here is that you're really trying to make up for

Â real problems in the coverage of your non-probability sample.

Â That's one of the big functions of it, and

Â it also can function that way in a probability sample.

Â