Welcome back. This is our last session together. By now hopefully you have seen all four modules and played with all major themes that we've discussed as we've gone through these modules. We wanna give a little bit of a bigger picture and some reflections on where the organizational challenges are for these issues and where the future directions are for this field. And we're gonna do this quite conversationally, just take enough the questions and kickin' around a little bit. Let's open with organizational challenges. The last module that in the course just before this session was how individuals might navigate the politics and organizational challenges, but that's just the individual level. What about the organizational level, what can we do as managers to set up the organization for success? In people >> I think one of the real challenges in organizations is the breadth of expertise that needs to be brought together in order to build an effective analytics function. Because, on the one hand you need an understanding of people decisions. I would be a little nervous if you just brought in a statistician to do this all for you. >> A little nervous. >> You need to have that background, right? But you also do need somebody who's comfortable analyzing data, understands sort of the statistical issues that you've been talking about in the so you need that expertise In our experience, getting the data is often a bear. I think increasingly you're seeing vendors selling entirely comprehensive systems. But currently you've got the personnel data in one system you've got the app for tracking data in a different system. If you're lucky, you can completely divorce them from any work flow data and so often, part of the time, you're stitching this together and saying somebody's good with SQL and kind of various other programming techniques that are necessary as well. So just bring all of that together within the organization [INAUDIBLE]. >> I think one of the big questions that really raises is, the function or the role of HR, Human Resources Department within the larger organization cuz a lot of this, it has traditionally being done out of HR Departments. The kinds of questions that we're talking about being explore, especially in large organizations are often HR kind of functions. But there's a lot of additional expertise that goes beyond, at least the traditional HR functions, that is gonna have to brought in and do you have HR work with other areas that are specialized in these? Do you bring them into HR? How I think that's a big challenge that organizations are gonna be, are thinking about in terms of being able to manage these volumes of data. Being able to collect really good data but then being able to process it, draw conclusions from it, and being able to kind of really change implemented as a result of those things. >> And I think the other thing that comes into that is being able to demonstrate the value as well, because what we were talking about is investment and costs. So bringing new skills, hiring new people into the function, and one of the things we've mentioned briefly is one of the areas of people in analytics, is being able to assess the ROI. Some of the stuff you're doing around managing people this is gonna need to demonstrate an ROI as well. So being able to provide quick wins and justify fairly quickly that investment. I think it's also something that people are gonna need to be able to do. >> In a classic organizational behavior kind of way, they also have to navigate how centralized these things are, is you want them working with the divisions, but at some point the divisions probably wanna go with their own capabilities, but having to do that at scale, and having to do something centralized and something distributed. It just depends like any other centralized function, you've got to decide how much is distributed and how much is centralized. >> Right, and it's certainly the case, just to underscore Matthews point, that the data are often not poor, right and are often very different, and fragmented, and scattered around organizations, so when we're trying to do analysis on, collect data for our own purposes, or research purposes. We're often asking organizations, okay, can you give us good data? And they'll say well we have a bit here and we have a bit there when we actually look at the data it's often incomplete. And HR has the same challenges in trying to deal with their internal data set. So trying to develop a more systematic way to collect good data in order to makes those measures again meaningful. And really being able to rely on the measurement that you have is gonna be a challenge that I think organizations are gonna have to invest a bit more. You can apply also, it's a great statistical tools to large data sets. But in the end, if those numbers are not meaningful and are not good performance evaluations, for example it's not gonna tell you anything very useful. >> This touches on what is another big organizational challenge and that is, what data is okay? What data is too sensitive? What can you take advantage of? And what data do you need to be careful about? So do you guys have thoughts on, I know there's no easy answers here, this whole issue of ethics around people on main thought is that it's a really, really, serious big concern for organizations that has to be taken very, very, seriously. >> I think the confidentiality of the data that you analyze and that you present to anybody inside the organization has to be absolutely a priority. Using aggregates rather than pulling out individuals is gonna be very important so it's not threatening to people if you talk about averages in teams or in their units provided they're of a certain size and you can't actually figure out who are the individuals driving that. Once you get down to anything that can identify individuals, you have to be really, really careful. So having very good data protection policies in place for that is gonna be very important. And then I guess people have to really, you really have to think about, do you let people opt in or opt out of having their data used? And again, a lor of data confidentiality and disclosure concerns are very, very high priority with doing any of this kind of work. >> Yeah, I mean I think it's, we've been pushing to some extent a fairly utopian vision throughout this course of look at all these great things we could do to make organizations more effective and make better decisions. There's obviously a dystopian alternative, right, which is we're moving to a world where everybody's being monitored everything that you do is kind of logged, the opportunity for more and more control of people in fairly negative ways is there as well so that's, I think that's going to be an ongoing tension for a while. I mean in terms of people doing this themselves, I think one of the big issues for me is gonna be about the kind of data that you use so, some of this data it's very clear why it's been collected and what it's been collected for. So if you look at the kind of stuff on applicant tracking systems or a kind of data performance data, a new personnel system. You kinda feel at that level people are probably not very surprised when your running analysis on that. Once you, the problem is there's so much other data that's around that you could pick up that people aren't thinking about locational data, about where they are off any kind of devices they have. People have started using the badge data, when people badge in and out of organizations out of the security gate to figure out how long people are spending in the building or so. I think when you were using data that was clearly collected for another purpose, I think that's when really these kind of questions of consent, particularly when it's then, if it's gonna be used to manage them. Like you say, whether people opt into the data and so on, I think it's gonna become more and more important. >> It reminds me of that quote about power just because you have it doesn't mean you should use it. I know an organization that has all of those data, and they're very sophisticated people analytics organization, they have all that location data and they've never used it because they realize how sensitive it is, and what the potential violation is. I do wanna give a little pep talk around that though in that I also know an organization that consults on unwearables and they have to navigate this exact issue all the time. They have to somehow recruit employees to participate in the wearables study and they do two key things. One, they make it purely opt-in, voluntary but then they, one give them feedback. And people love feedback about how they compare to other people and all the details about their lives and the patterns in their data. And so they provide that as kind of a reward, essentially and the second is that they guarantee that the organization doesn't get the individual back to your point about individual versus aggregate. But the supervisors that essentially don't get individual level data that they can hack, so with those, what they find, with those two with that reward and that guaranteed, they're able to actually pull people in quite happily, to participate. >> Yeah, I mean there's, in management there's this ongoing debate about technology bureaucracy all sorts of things. So it's got a constant theme throughout the literature, which is that any of these things can be used in two ways either in an enabling way, which is helping people do their job better or empowering people, or a coercive way, to kind of exert control. And I think you're going to see that playing out, definitely in people analytics. There are lots of ways we could use this and the simplest, you do figure that kind of the simplest thing for anybody doing this in practice Is the kind of Wall Street Journal test. This ended up on the front page of the Wall Street Journal, how would that work out? Or is this something, are you comfortable presenting the results of your study in an all hands meeting, kind of a town hall meeting at work. If yes, go ahead and do it. If no, pause, think twice, figure out are there different ways to do it. >> But also even if you're comfortable, the people whose data are being collected may not be as comfortable. So I think one of the grayish areas, most gray areas, is kind of the data, as you mentioned earlier, that kind of emerges as a byproduct of other things that we're doing. It wasn't meant for that process, so we sometimes call it the breadcrumbs of big data, these kind of be it the breadcrumbs that you leave behind when you're doing something else. And so particularly in terms of the collaboration stuff on the communication data, right? So also, so whether it's emails, or phone calls, or that kind of data that you're doing it for one reason. And people are collecting that data, monitoring their data, and looking at those patterns. Now there's ways to deal with that to some extent. Looking at the content of emails is very, very different from looking at the number of emails that get exchanged between people, for example. But even if you felt comfortable presenting analysis [LAUGH] to your management board, the people whose days would be collected may still not feel comfortable. So, these are very, very sensitive issues that need to be thought about at multiple levels. >> Absolutely, and kind of when we do these studies, right, I mean we are subject to a lot of oversight. So every US university has its institutional review board where you have to justify your study, talk about how people are. And it's very stringent in terms of a lot of the things you said about not identifying individuals and all these sorts of things. Your average corporation has none of this, doesn't have the same legal responsibilities, and so on and so I think how they start to think about what the boundaries are, I think is. >> Probably, just holding themselves, holding yourselves to the highest standards is gonna be the sensible way to go at this point. >> And the field is still being created if people are responsible about this and organizations who are doing this at the cutting edge are giving good examples then it's gonna go one way. If people are irresponsible, we have a bunch of bad examples that can go the other way. >> It almost certainly can go both ways. >> [LAUGH] >> I mean, it's- >> That's right, whenever you're talking about utopian versus dystopian we're gonna get both of those, aren't we? >> Yeah. >> That's right. >> So you could say choose your employer carefully, but obviously, we don't all have a lot of choice in that. Well, but it's a good thing to emphasize all the same. Okay, we wanna close with some thoughts on future directions. We've tried to give people a sense of the state of the art and a few basic application areas, that kind of every organization is participating in. These are also areas we've seen companies take up quite quickly, we give lots of examples in all of these areas. Where do you think we're going? What's the cutting edge here right now, and where do you expect it to grow over the next few years? >> I mean I think some of the most interesting stuff for me is coming out of computer science. And I think a lot of the stuff we've talked about analytically very simple, right? You've got fairly clean data and you see how one thing correlates with another. When we talk about, we keep talking about all this data that organizations collect, a huge amount of it's unstructured, right? So you've got a lot of documents, you talked about the content of emails, you've got video, voice recordings or whatever. A lot of thought is going on into how do you analyze those and can we use that to understand so people. Actually the content of people's communication to understand who's communicating effectively, who isn't, what can we teach people about how to communicate. I hear there's some organization that is trying to create an algorithm that can use a sample of somebody's voice to understand how they'll be perceived by customers from a call center. >> And so there is, yeah, I think there's a lot of the forefront is a lot of this unstructured data and trying to make this rich data more structured. It'll be very interest to see how that plays out. Obviously, I think one of the concerns with that is the other things we've been talking about. Those algorithms are real black boxes, right, and so you stick in a sample and it comes out saying yes or now and I think you become even less comfortable about, do we really have a sense of what's going on in there? But there are always possibility there as well. >> But the danger is, people are selling them as, it's almost a snake oil era that big data has provided. Because they act as if look, just give me all these things and I'll tell you the answer, it's like that is a really hard task. And it's a really big job and it's gonna be a while until that can be done well. And I'm worried that between now and then it's gonna be sold to lots of people who don't realize that it's not being done well. >> Yeah, yeah, I mean that's true, I think another thing that's really exciting and promising about this is not just big data but smart data right, so I mean yes there are increasingly. Huge data sets everywhere, we can collect information on just about everything. But what's really exciting about a lot of these data is that they can get really, really fine grained into what people are actually doing or where they are or how they're communicating in a way that is much richer and more fine grained than any other data we've ever had before. And so what we can do as a result of that is not just Be able to sorta correlate outcomes to things like how many training sessions you went to. But we can actually really get much more fine grained about what were your behaviors, but those training sessions were trying to influence and can we actually track of those behaviors? How much in fact are you meeting with, talking with the people that you went to the training session afterwards, for example, is that one of the benefits of training? What were you actually doing in those training sessions that, how did, how did that content track. So, and again, you have to be very careful about imposing too much into people's lives and the confidentiality issues that we were talking about before. But that is also the promise of statue is that they can get you to much better understanding of whats driving out cons and these kind of, sort of more course measures that we've used up til now. One of the other things that I think is really exciting is that once you have a theory, or a relationship that you think exists, based on analyzing these kinds of data. I think companies are starting to use field experiments much more and so the idea that you think you know what's going on, and then setting up a kind of pretty rigorous way to test it, inside a large organization. Finding some part of your organization or two parts that you can compare and then running the experiment in one part and then not running it in the other part. I think that has a lot of potential a lot of companies are starting to do that as an outgrowth of kind of what they're able to do, the findings that they're able to generate through these kind of people analytics techniques. >> Which is a huge development, because companies have been so reluctant in the past to run experiments on employees. >> Right. >> Understandably perhaps. >> [LAUGH] >> Right, but in contrast to the marketing side of the where it's the way they've kind of grown up doing business, the AB test and with the Internet now it's trivial to run these things. But I think that the HR folks, the employee folks, have been influenced by what's going in in marketing and people increasingly appreciate you need the field experiment to really understand and so maybe there are ways we can run some experiments with our employees, and in fact, we are seeing organizations do that, which is wonderful, >> This may return to where we were at the beginning of the discussion. But, I think it really plays into this kind of centralization, decentralization thing, because I think this spreads through the organization. I mean, the infrastructure to run these analyses ro do these field experiments and so on does require some sort of centralized competence in kind of the trend of the last 30, 40 years, of being more and more of decision making and authority going out to line managers. And so it will be interesting to see to what extent this is able to build up again kind of a sense of competence and a sense of expertise. In the center, which can then start to move these topics forward. >> Let me add one more, and it's mostly related to Martin's work in these modules. But it cuts into some of ours as well, and that is better evaluation of individual contributions to team efforts. Really is everything, we've just been so individual focused historically, and by that I mean the last, all of human history, right? We miss the interactions, we miss the subtleties of the positive consequences of someone working with the team that kinda gets below the radar or the negative consequences. And I know this is one of the things that people grapple with a great deal in the sports world. And I'm optimistic that some of the techniques from that world are gonna flow down into other organizations. But the collaborative tools that you talk about give us some insights that we've never had before and the hope is that we can start identifying. The individuals who make contributions to teams that people don't appreciate and the behaviors that people exercise or don't exercise that make positive contributions to teams. >> And hopefully, also how people actually experience the workplace and whether they feel pressured by some of those expectations or they could contribute more and they don't. So where do you intervene to help falls to their skills, or to train them better, or to maybe reduce the burden on them if it's not working so well. So the fact that we can really get much deeper into what people are actually doing in the workplace in all these ways has a lot of positive potential, as well as some dangers that we need to be very careful about. >> Yeah, I think beyond Isolating the impact of the individual maybe also just starts to take the team seriously as a unit. So there's always this risk that it's, okay, but we really want to know what each individual's contributing. >> That's interesting, okay. >> Almost by doing that, you start to disrupt the team dynamic. >> That's interesting. >> And maybe sometimes we're actually better off just saying teams work? >> And also developing measures that can be good measures of team outcomes from what are the behaviors of individuals that actually contribute to the team outcomes and all that is stuff that we should be able to do now. >> Yeah, we certainly aren't gonna be stuck for questions any time soon. >> [LAUGH] >> No, exactly, exactly, well this has been fun guys. >> [LAUGH] >> Till next time, thank you for being here. >> Thanks very much.
