We've talked a lot about analytics in this course and that's the nature of it and that's the heart of it and that's what we wanted to emphasize. But it would be silly and unwise of us to not emphasize the organizational challenges associated with analytics. I'd go even farther and claim that effective people analytics. Is more of an organizational challenge than an analytics challenge. So while we emphasize techniques and principles associated with analytics, let's leave you with some techniques and principles associated with the organizational challenge. So the main prescription will flow from the simple idea, one dominant theme in talking with firms and talking with the analyst in firms, and that theme is no black boxes, but we're going to unpack that and tell you what we mean. The specific prescriptions that come from that no black boxes theme are to be transparent, to embed yourself, and to share control. So, we'll unpack each of those to send you out with some tools you can use on making sure that you're not just a good analyst that you're an effective analyst. So, first be transparent. So we know quite a bit about the way decision making is received by those who are affected by decisions. There's a long literature now on procedural fairness, Joel Brockner has been one of the leading researchers on that. And this article, for example, in the Harvard Business Review he covers the elements involved with procedural fairness, and transparency is one of the main themes. People are more comfortable with processes when it is transparent, what is going on. An organization that's taken that to heart is Google. It's one of the main themes in a recent book by Laszlo Bock, the head of people operations at Google, who emphasizes it throughout And it's critical to their work on Google Analytics. An example of the extent to which Google takes transparency seriously is their TGIF, their weekly TGIF, which oddly takes place on Thursday, in a very Google way. But every Thursday they have a gathering, originally it was with the two founders, and often the founders are still there. Sometimes it's the CEO. When those guys can't make it at some very senior executive and all the employees that can make it are crowded into one of their cafeterias, they watch by remotes around the world. And they can ask the CEOs, the founders, the senior executives, anything they want to. And they do ask all kinds of things, and the guys stand there, the women stand there, and they take those questions, and they give responses, and it breeds a culture of transparency. And it emphasizes the importance the organization places on transparency and that is something they believe has helped them to use analytics more successfully in the organization because people trust them. They understand what's going on. They're more willing to embrace the analytics that drive the people decisions. A second prescription that comes out in this no black boxes theme is to embed yourself. And to illustrate this, I wanna start first with the research of Bob Cialdini on Influence where he unpack the six principles that make people persuasive. And one of those, one of the most important is liking. And he decomposed that further to similarity and ultimately finding that a fundamental driver of liking is similarity and then liking drives influence. The key point is that people are more readily influenced by people they're more similar to. We may like this, we may dislike this, but this is human nature and it's an important lever to pull if you're trying to influence others in organizations. So, Chaldini suggests find or create sources of similarity. And, an important way to go about that is to embed yourself in organization you're trying to influence. You don't want to be seen as other, someone on the other side of the wall. You want to be seen as one of the people you're trying to influence. Alec Scheiner is the president of the Cleveland Browns. Before that, he was with the Dallas Cowboys. He's one of the most influential executives in professional sports, certainly within the National Football League. Long an advocate of analytics, he has been fighting the battle of influencing organizations through analytics for years now. And this is how he put it. He said, embed yourself. He gave the example, we hired an analytics guy, Ken Kovash. He's in the audience here, I'll show you a picture in a second and I always tease him because he wears Browns t-shirts everyday to work and I'm like you didn't do that at your old job at Mozilla did you? You just like wearing t-shirts to work everyday and pullovers? But there's a point to it, he's dressing like our coaches and our scouts and they feel more comfortable with him. If he came in everyday in a suit and tie he'd be like a foreigner in their land, and I'm gonna hit on this again and again. If people don't trust you then they're not gonna listen to you. They might act like their listening to you, but they're not. First you have to build their trust, and one way to do that is really learn what they do, and to do that you just have to embed yourself. Work near them, go out with them, see what their day to day is, see what their challenges are. That's really what we spend most of our time worrying about. So later I got a picture of Kovash and there he is in his Browns sweatshirt. Sure enough, being like the coaches, being like the players. Right after Shiner gave this comment, I walked, this was at a conference, I walked out of the conference and I was walking alongside another analyst for another team, and I asked him what he thought about the session. And he thought the session was okay. But he called out exactly this comment. And this analyst told me, lately when I've been going from the office to the playing field to watch practice, I don't change. If I change I have to go to the locker room, it takes me ten minutes, it slows down my day. But walking out of that meeting he said, I'm gonna start changing clothes. I'm gonna start putting on the sweatshirt, because it's gonna make me a more influential analyst. There's some version of this. We don't all work for football teams. We don't all put on sweatshirts. But there's some version of this that will make you a better analyst if you figure out ways to embed yourself with the size of the organization you're trying to influence. Finally, share control. This is a big one and a hard one for us especially when you know you have the right answer because of your analytics, you still somehow have to find a way to share control. On some research I've done with Berkley Depvorced, a PhD student here at Wharton and Joseph Simmons, a long time friend and colleague here at Wharton. We've found this notion of algorithm aversion, that people are averse to using algorithms so often from analytics, that's what we've come up with, especially when they see the algorithm's error which inevitably they will. So we've done a little bit of research on this to try to figure out how we can break it down. And we find that across a wide range of tasks, people prefer judgement, human judgement, to algorithmic judgement. This is even true when algorithms are better. Even when they know they're better, they prefer human judgement. And the main reason for this is that they're more forgiving of error by humans than by algorithms, all of this is challenging. This is a big challenge for those of us who are trying to do better analytics, which inevitably leads to algorithmic judgement. So what do you do? This is the last thing we find. People are more tolerant of algorithmic judgement when they have some input, even when that input is minor. So we ran a study, I'll show you quick results from our study, where we asked people you can either use a model, algorithm based on some analytics that have been done, or you can use human judgement. But we gave them four different conditions. The first, they had to choose between the model unaltered, the model as is, as provided by the analyst or human judgement and we found that about 47% wanted to use the model. This was in a predicted task, the details of which don't matter right now. The other three conditions, we allowed them to take the model and adjust it a little bit and in every other respect the study was the same. So in one of those conditions we allowed them to adjust it by 10%. And what happened? The input, the intake of the model, the uptake of the model, the interest in the model went up almost 50%. From 47% it increased 24 percentage points, to 71%. Interestingly, so big update by letting them have participation and by giving up some control. Interestingly, we looked at two other conditions. Reducing the amount of control that they had what difference would it make? If we allowed them to just pay only five percent what happens? Still 71% uptake. What if we restrict their input to just 2% so they can move the model but just barely, what does that do to their intake, uptake? It doesn't change it. What we find is as long as we give them some control, some input into the final decision, they're much more willing to lean on the algorithm, to lean on the model. This is underscoring this idea of giving up a little big of control. Now it might make the algorithm a little bit worse. But maybe that's a price we're paying if it gets the interest in the algorithm higher. That's definitely what we've found in our studies so far. And we're beginning to use this technique in our work with organizations. One example: a local graduate admissions office, where we blend experts and analytics. Three stage process. And the first stage, experts evaluate each applicant. So this is selective judgement. But in the middle stage we use the decision model. Algorithms, we use analytics to crunch those evaluations to recommend the optimal class. That might be where we'd stop, but because of what we've learned on giving up some control, we knew there needed to be a third stage and in that stage the experts get to review and revise those recommendations. So the algorithm doesn't get the final say, the experts get the final say. And even if they don't change very many of those recommendations, the ability to change, the leeway we give them makes them much more open and much more interested in the analytics and the algorithm that comes from the analytics. So to wrap up. On the organizational challenge side, the major theme is don't make it a black box. No black boxes and in particular be transparent, embed yourself in the organizations and as much as it might hurt you to do so, share control. You'll be a more effective analyst if you do.
