So in this segment I want to talk about hiring, the process of getting people into jobs. If you think about hiring in most organizations, it's pretty clear that sometimes we do it well and sometimes we do it badly. If you Google online you'll see a lot of statistics, about maybe half of all new hires fail. It's never been clear to me where they get those statistics from, but it's certainly the case that we can see in most organizations. We hire a bunch of people, some of them work out, some of them don't. A lot of people leave their jobs fairly quickly which suggest this was not a good match. And so, it's done fairly imperfectly pretty much everywhere. When you think about what hiring is there are a bunch of different ways of thinking about it. So often we think about it as a match. It's almost like dating. It's finding the right chemistry. And that's usually influential in how people actually hire. From an analytic standpoint though, the way that we think about hiring is about predicting performance. We have a lot of people who are applying to the job. What we want to know is who's going to perform best in it? And so all of the effort goes into figuring out what predicts high performance, so that we can then screen on those attributes. What I've got here is a list of different things that companies use in order to try and screen applicants, to try and understand what their performance is. So we see things like job knowledge tests, cognitive ability tests. use personality test, reference checks, structured interviews, unstructured interviews, work samples, integra test, a wide variety of things that can be used to try and get at this question, who is going to perform well in the job? Question that we have to ask then is which of these are best? So, if we're thinking about assessing candidates. Which of these should we be using and which of them shouldn't we be using? So as you look at that list. What do you think are some of the thing that are gonna be most effective at predicting performance? Had a chance to look at it? Think about it a little. Okay, well let's see what actually works. So what I've got here are a series of correlations. So I don't know how much you know about statistics, and one thing that I'm going to say repeatedly during this module, is that if you haven't taken a course on statistics and you're interested in this topic. I really strongly recommend that you take a couple of introductory courses in statistics, because they're sort of the complement to what we're doing. You want to understand the people side of analytics, and how to apply analytics to people but statistics just gives you an understanding of some of the ways of interpreting the results. And for those of you who haven't done this recently, correlation is basically a measure of how closely two variables move together. So correlation of zero means these things are totally unrelated to one another of correlation of one means they move together perfectly that they could almost be the same thing. And where they fall in that spectrum tells you how closely they're correlated. And so what you see in this graph is a variety of correlations. The largest correlations are the things that are doing the best job of predicting performance. Correlation of .5 is quite strong, it's, again, not 1. It's not a 1 to 1 correspondence, but it tells you this is quite good at predicting performance. When we get down to kind of, .2, .3, that's a lot weaker. And so, what we've done here is we've listed in order how the various different assessment techniques actually predict performance. So if you got it right, congratulations. Give yourself a pat on the back as long as you're not carrying anything heavy or something like that. If you got it wrong, I'm sorry, but better luck next time. Anyway, if you go through what you see, is that there are a number of things that really do predict performance quite effectively. So more than anything else, perhaps not surprisingly is work samples. And so some companies, when they hire, they ask people to either produce, or perform something like the work that they're going to do. So for example, technical writers. Maybe you'll give them a description of your product and ask them to give us a sample of writing about it. I heard companies hiring analysts, maybe we'll send them a spreadsheet of data and ask them to analyze it. Just getting a sense of, can they perform the job they're being asked to do. Another thing that is surprising, I would say affective, is congnative ability testing. It turns out that actually, general intelligence is really quite predictive of performance in a very wide range of jobs. And so some organizations will actually have candidates do something that looks like an IQ test, if intelligence is something they care about and drives performance, it's the easiest way to measure it okay? Third thing that we've seen is very effective is structured interviews. What's the difference between a structured interview and an unstructured interview that comes down at the bottom? An unstructured interview is really when we meet somebody, and we talk to them, and we try and get a sense of Who they are, what they're like, what their strengths and weaknesses might be. We all think we're good judges of character. It turns out, we're not. So, I don't know if you've read, it's a marvelous book. Thinking Fast, Thinking Slow, by Daniel Kahneman. You get a chance, you should really look at it Carniman is a Nobel prize winning psychologist, which is quite a neat trick, since there isn't actually a Nobel prize in Psychology. He won it for Economics. He has done a lot of work on biases and characteristics. He tells this lovely story about the first job that he ever had. So he was working with the Israeli defense forces, and his job as a psychologist was to try and assess candidates for officer training. And so they had this exercise where they would get these candidates carrying this large log, and say, okay, you've got get all of you candidates, and this log, over a six foot wall, and neither you nor the log can touch the wall at any point, and the log can't touch the ground. And the idea was that psychologists would stand around and watch them as they try. And so you'd see some people would try and organize people that come up with the ideas. So coordinate, other people might hang back, some people get very annoyed when their ideas got ignored. In other cases you'd see when it was going wrong. Some people would give up, but others would really step forward and start trying to organize people. They figured, based on this, you'd get a really good sense of people's leadership ability. And so, they would rate people, and make recommendations about who should go to officer's training. The interesting thing about the story is that they actually knew how good their assessment were. They could see the end as people went through officer training who actually turned out to be good at this. And they discovered there was almost no correlation between what they thought based on kind of what seemed like a good set of test, and how people actually performed. What was really scary to Kahneman was that even though over time they learnt that this test was in no was predictive of performance, nonetheless for each individual they still felt very confident in making a prediction. Okay we know in general this doesn't work, but we know this guy is going to make it. This guy definitely not going to make it. And he calls this the illusion of validity, a sense that we think we know much more about people than we actually do. And the challenge with on structured interviews when we sit down and we talk to people, and try and figure out what they're like is really giving into the illusion of validity,right, we don't know learn very much, but whatever biases, prejudices we have. They fill up the whole room in our mind in terms of actually judging them. So unstructured interviews are found to be really pretty bad at predicting what people are going to do and how they are going to perform. Structured interviews, on the other hand, are when actually interviews are designed to assess specific characteristics of the individual, and get at whether or not they have the attributes needed for the job. They're found to be fairly predictive. On the other end of the scale, I think something that often surprises people is how ineffective reference checks are. We have a sense that if we ask people who know the candidate, they should really be able to tell us how they perform. There are a couple of problems with this. I mean there are certainly legal issues in the U.S. now, which is companies are very afraid of giving negative references because they might be sued. On top of that, and certainly having read a lot of reference letters over the years. You realize that most of what drives the quality of a reference is actually the person writing the reference, right. How happy they are to lie outrageously on somebody else's behalf to help them get a job, and when you're trying to predict performance, and most of the variations in your measure is being driven by the person writing the reference. Rather than the person the reference is about, is going to be quite a poor measure of their future performance.
