So a second dimension, of thinking about how we can bring analytics to Staffing, is thinking about how we evaluate the different ways that people can be staffed. So, if you think about most jobs, I have here a representative organization, it's a pyramid. How do people get into these empty jobs, designated by hollow eggs? Well, see one way that people enter them is by being hired. Another way is by being promoted. And a third way that people might enter jobs, is kind of sideways. Now, if I'm a manager and I'm thinking about staffing. A question that I really care about is what are the trade offs towards this. Should I be doing more to promote internal development and promotional mobility, is that money well spent? Or would it be better off spending that money to try and beef up our recruitment efforts? Which of these is like to be more effective? So this scenario where I've done some work and kind of give a little more detail how you do this analysis. So I took a bunch of data from a division of a large investment bank. Basically, just told me how people were performing in their jobs and how they were paid in kind of snapshots year by year by year. And so you could go through this and pull out for each person, okay, how did they get into that job? Were they hired into it? Did they come from a lower level job in the organization, being promoted? Do they come from somewhere else in the firm, were they being transferred? And then see which of those shapes, the various performance metrics, as well as how they're paid, okay? Now in doing this there are a bunch of things that I was worried about. So one is, obviously everybody gets hired at some stage, so kind of worry about what happens after somebody's been hired a while and been promoted. So let's only look at how people entered their current job. Another big issue is you've got a lot of different jobs here, okay? So if I just take everybody who's been hired, and everybody who's been promoted and compare them. Well that's really going to be conflated with the fact that some jobs are likely to hire more and other jobs are likely to promote more. And so really I wanted to use very detailed controls, talked a bit about this in multi-variable regression. You kind of strip out those job-level effects. And so I could conceptually think about comparing people in the same job. What are their outcomes based on how they entered the job, and kind of equivalent to that, I also wanted to only have jobs that could be entered in multiple ways. And so, two things I wanted to do. So, first of all, let's drop all the lower level jobs that you can't enter by promotion. And in addition, I also wanted to drop anybody who I didn't see entering the job. Also one of the big concerns is people particularly over time their performance changes. And so I only wanted people I knew when they entered that job and how they entered that job. So kind of clean the data set first and then look at these relationships. In this case I saw quite a lot going on. So first off I saw differences in performance. So the people who are hired into the job perform much worse than the people who are promoted for about the first two to three years. Much more likely to get poor ratings, much less likely to get the highest ratings. You saw over time their performance converges. And so it's not necessarily they are hiring worse people. But it kind of really said something about how long it's taking people to get up to speed in this organization. Then you can't just transition and get off to a running start, okay? People need to learn even when we're coming and high levels require experience from us. The other piece of it that I saw that was quite interesting. So even though there are performance differences, people who were hired were performing less well. It was also the case that people who are hired were getting paid more, okay? So we're paying people a lot more, and being able to perform worse doesn't sound like a good deal for the organization. And these were differences that were actually much more persistent than the performance differences. And we can dig into that, it's possible to see that a lot of that has to do with the differences in who you're hiring, when you hire from outside you don't know much about people's performance. You tend to want to make sure you have a good resume and so these people coming in from outside then have more education, more experience as an employer, you have to pay more for that. So I certainly thought this is a fun study to do but I think it has some practical implications right? If I was in an organization I want to know this, why? First, okay, maybe we should think more seriously about our internal development. On the margin, hiring looks quite expensive. The second thing that it really tells me is wow, it's taken a long time for our hires to get up to speed, so what are we doing about onboarding? How do we help accelerate that process of learning? That's something we want to put in place, and maybe that's something we also want to assess, seek on all these performance ratings, converging more rapidly once we've done that. So this kind of evaluation of staffing options can tell you a lot. We have another example of doing this kind of work which was done by JR Keller who's at Cornell. He did this when he was a PhD student at Wharton, we're very proud of him. He looked at another difference in the way that companies staff jobs. Obviously, a lot of people move jobs inside firms. Well they do so often in two very different ways. And so kind of a stereotype is this idea that kind of your manager walks in one day and says congratulations you've been promoted. That happens a lot but the other thing that can happen is that jobs are posted. So somebody puts together a formal job description, they stick on a website, maybe they put it up on bulletin boards and say we need somebody to fill this job. If you're in the organization you very welcome to apply and then we'll go through and we'll see who does best. The organization is looking at, they did a lot of filling jobs both ways. In fact, I think most organizations now use both these mechanisms. Well, from the HR department I'm kind of curious which of these should I be encouraging? Which tends to work better? And so he looked at all of the outcomes of people entering their jobs through these to see are there differences. And he found they really were. So what he found in particular were some real advantages of internal posting. First of all, it enables people to do more unusual things. And so the career paths that most people made through internal posting were more unusual, which kind of makes sense. It's people from throughout the organization saying I'm not clear on what I'm to be doing now. That looks more interesting, let's go try that. What's more intriguing was that he also found that performance was systematically higher after people had moved jobs through posting versus the manager just sponsoring them for a job. One reason for that, potentially, is we're looking at a wider pool of applicants. And so we're taking the best person from a much greater pool, people who we wouldn't necessarily have thought of for the job, but are interested and turn out to be very good at it. And certainly, we see this kind of way to pool. But the other thing that was really interesting was that people who had been in the same group all along. And got promoted into a job through posting also performed higher than if they'd been promoted into the job through sponsorship,okay? So, it's not just about getting a richer pool, and I think the story that seems most plausible here kind of back to hiring. So I talked there about kind of more structured process doing better. So kind of posting some more structured processes as well. And what we think's going on is the managers, when they're forced to think about it, when they know they're going to have to justify that decision to the people that they didn't promote. Take a more structured decision making approach and probably do it better. The same time we saw some things were good for workers or bad for the organization, depending upon your perspective. We also so paved change depending on how people entered the job. Looked holding everything else constant, but when people moved jobs by posting, they got more of a pay raise. We think that has a lot to do with negotiations inside the firm. When you've established you're the best candidate, you have a stronger negotiating position. When you've been forced to apply through a market, you may also feel more empowered to actually go out and ask for more money. And so we see yet again how people are staffed matters. And the value of within your organization trying to get a handle on which of the processes are better, which should we be encouraging, and which shouldn't we. Of course when we think about making those decisions one of the things we worry about a lot is what's really driving these relationships. Does this mean that if we do more posting we'll see better performance. And that gets in to issues of causality and I want to spend a few minutes in the next segment just talking about how we think about causality and where we need to worry about it.