Hiring 3: Using Data Analysis to Predict Performance

Loading...

From the course by University of Pennsylvania

People Analytics

ratings

University of Pennsylvania

People Analytics

ratings

Course 3 of 5 in the Specialization Business Analytics

People analytics is a data-driven approach to managing people at work. For the first time in history, business leaders can make decisions about their people based on deep analysis of data rather than the traditional methods of personal relationships, decision making based on experience, and risk avoidance. In this brand new course, three of Wharton’s top professors, all pioneers in the field of people analytics, will explore the state-of-the-art techniques used to recruit and retain great people, and demonstrate how these techniques are used at cutting-edge companies. They’ll explain how data and sophisticated analysis is brought to bear on people-related issues, such as recruiting, performance evaluation, leadership, hiring and promotion, job design, compensation, and collaboration. This course is an introduction to the theory of people analytics, and is not intended to prepare learners to perform complex talent management data analysis. By the end of this course, you’ll understand how and when hard data is used to make soft-skill decisions about hiring and talent development, so that you can position yourself as a strategic partner in your company’s talent management decisions. This course is intended to introduced you to Organizations flourish when the people who work in them flourish. Analytics can help make both happen. This course in People Analytics is designed to help you flourish in your career, too.

From the lesson

Staffing

In this module, you'll learn how to use data to better analyze the key components of the staffing cycle: hiring, internal mobility and career development, and attrition. You'll explore different analytic approaches to predicting performance for hiring and for optimizing internal mobility, to understanding and reducing turnover, and to predicting attrition. You'll also learn the critical skill of understanding causality so that you can avoid using data incorrectly. By the end of this module, you'll be able to use data to improve the quality of the decisions you make in getting the right people into the right jobs and helping them stay there, to benefit not only your organization but also employee's individual careers.

Introduction to Professor Bidwell0:30

Staffing Analytics Overview2:04

Hiring 1: Predicting Performance8:50

Hiring 2: Fine-tuning Predictors9:07

Hiring 3: Using Data Analysis to Predict Performance7:44

Internal Mobility 1: Analyzing Promotibility4:55

Internal Mobility 2: Optimizing Movement within the Organization8:26

Causality 15:11

Causality 26:47

Attrition: Understanding and Reducing Turnover10:25

Turnover: Predicting Attrition7:52

Staffing Analytics Conclusion0:49

Meet the Instructors

Cade Massey
Practice Professor
The Wharton School
Martine Haas
Associate Professor of Management
The Wharton School
Matthew Bidwell
Associate Professor of Management
The Wharton School

So how the people actually go about addressing these issues?

So [COUGH] there variety of different things that you can do kind

of the simplest is just to take your best and worst performances in the job.

And compare them on a variety of characteristics to see which

characteristics are best predict performance.

As you do that, one of the things I do think you wanna do is

what we call test the statistical significance.

Again, please think about taking a course, an introductory course in statistics.

These tests for statistical significance are basically trying to see,

you know, is this a big difference or

could this just be explained by the fact there's random variation in any data.

So there's kind of the very basic.

I think better than that is to try and

be a bit more systematic about who you compare.

So, for example, like I said, we expect the time somebody has been in the job to

have a big effect on their performance.

So you want to compare people who've had similar amounts of time in the job and

look at what characteristics shape who's a better and

worse performer within that cohort.

Also try and just look at people who are on the same job together.

Even better than that, and I think what most organizations would look to do will

be what we call multivariate regressions.

I'm not sure how many of you know much about this, so

I'm just gonna give a very brief conceptual introduction.

It's something you can't actually do in Excel.

If you Google it there bunch for other statistical packages.

It's very useful for this kinds of.

So the basic idea of what it's doing is,

if you plot out your data set, so, plot out,

see here y would be performance, x could be for example GPA.

And so each dot is a person right, and

you've got on the one hand their GPA, on the other hand their performance.

And you get the kind of the scatter plot like this.

And you want to understand, is there a relationship,

how much does a higher GPA actually predict performance?

Conceptually, what multivariate regression is doing

is it's just finding the best fit line through this data.

The line that you would draw that really summarizes this relationship.

And it has three very nice characteristics for doing this work.

Okay the first is usually it doesn't do this graphically, but

it spits out an equation that looks something like this.

So here maybe performance is 3.2 on average,

plus .2 times whatever the GPA is.

So it gives you a way to quantify the effect of this variable.

The second thing that's nice about this is

that it also tells you the level of statistical significance.

So is this really just a gap, if we throw a bunch of dots at a piece of paper,

we're gonna be able to draw some line through it.

You know, is that what's going on here or

does there really look like there's a relationship?

The third reason the multivariate regression is useful,

particularly in this context, is you can have multiple different x-variables.

So, I'm not going to try and

draw what this would look like in multidimensional space.

I really don't have that artistic talent.

But you could imagine, you know you don't just have GPA.

Maybe you have their level of related work experience, and maybe you have

how well they rated on a job knowledge test, and various other things.

And you want to try and

disentangle how each of these variables shapes performance.

That's what the regression will do, will kind of draw this best-fit line through

this multidimensional space, and you'll end with something that looks like this,

where it tells you okay, has an effect on GPA.

So GPA affects performance, in addition so

does experience and so does the job launch test.

And this is how much.

And so

it's that ability to untangle multiple different things we know about people.

Figure out which of those have the strongest effect on performance and

which of them doesn't really matter very much.

Where regression is really very strong.

And so I think that's probably the way that most people if they had the time

would go about doing this.

Probably best practice would be also to try and think a little bit about this

selection process that we're not looking at what this relationship looks like in

all of the people that applied, but only the ones that we actually try to hire.

This is quite challenging cuz obviously you don't have the performance data

on all of the people that you didn't hire.

There are various statistical techniques for actually correcting the sample, so

you can correct for that.

You can also potentially correct for the fact that not everybody stays, and so

the sample of people for

whom you have this data is also selected on the other side by who's left.

So there are things like Heckman selection corrections that do this for you.

They're quite sophisticated.

You want to take your advanced statistics course for that, and

they also have a number of problems in terms of they're kind of quite sensitive

to some of the statistical properties.

So most cases I think people will stick with multivariate regression.

I think a question that a lot of people are interested in as they do this is,

okay so we're kind of thinking we're gonna use an algorithm to give us a sense of

who to hire.

Really, we're gonna let a computer tell us?

Surely, we should have a better sense of this.

How good are these algorithms?

So, there's some bad news here.

So, the bad news is these algorithms are very far from perfect.

The combination of various effective means, work samples,

cognitive ability tests, all these sorts of things, most cases probably

don't predict more than about 30 or 40% of the variation in performance.

So we're not that good at really identifying

who's gonna be your best performer.

That's the bad news.

The worse news is that they're probably still better than people are.

There was a very nice study done by Mitch Hoffman at the University of Toronto and

some colleagues where he looked at data on hiring into call centers.

So I mentioned earlier attrition is often one of the things that you're

worried about.

You're really worried about that in a call center because most of these call center

will take one, two, three,

four weeks to train people before they ever let them answer the phone.

And then turnover is very high, a lot of people leave after three, six months,

something like that, and so it's hugely costly.

And so being able to drive down turnover is a big influence on cost, so

it's one of the things they really screen for heavily.

There are a number of vendors that help them with this, and so

Mitch had data that looked at how these performance screens,

which are algorithmic.

We give you a bunch of tests and then we spit out a score,

how they did with predicting turnover.

And so in particular what these screens do is they just advise the manager,

they say this person is a good risk.

This person's a poor risk.

We found a couple of things.

Firstly, when companies adopted these screening techniques,

then their turn over dropped.

So when managers had more information on who is likely to turn over,

they did better in terms of hiring in terms of having lower turn over.

The thing that was more frightening was that managers also had discretion.

So they got this advice but they didn't have to follow it.

But what Mitch found was that when they used their discretion more,

turnover was higher, and so I think the most obvious interpretation of this

is algorithms are not great but they're still better than we are.

You know we get back to Daniel Carnamen carrying his log around, right?

The idea is that algorithms are not perfect, we're worse.

And so there's huge potential in this space for data analytics and

people analytics to improve how we go about hiring.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2018 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play