0:00

Hi, in this last lecture about Markov processes, what I want to talk about

Â is exaptation. I want to talk about how we can use the Markov model in context,

Â and for problems we never would have thought of. And we do this in two ways.

Â First I'm gonna talk about taking it whole hog, taking the entire process and

Â modeling other things, just like we modeled the process of states becoming

Â free or dictatorial. Second, what I wanna do is, I wanna just take part of the Markov

Â model, its transition probability matrix, and use that to understand some things

Â that are really kinda surprising and interesting. So let's just remind

Â ourselves of what a Markov process is. Fixed set of states, and then there's

Â fixed transition probabilities between those states. Now if it's possible to get

Â from any state to any other through a sequence of transitions, then remember, we

Â have this Markov convergence theorem that says the process is going to go to a

Â unique equilibrium. So history doesn't matter. Initial conditions don't matter.

Â All that sort of stuff. Let's think about where else we might apply that. So we, one

Â place right away is voter turnout. So you can think about, there's a set of voters

Â at time t, and there's a set a of non-voters at time t, and what we can do is

Â draw a little matrix and say, how many of those are going to vote at time t+1,

Â and how many of these are not going to vote at time t+1? And it could be that 80

Â percent of voters at time t vote in t+1, twenty percent don't. It could be of

Â nonvoters that 40 percent vote and 60 percent don't vote. This would be our

Â Markov transition matrix and if you apply this, what you'd get is a unique

Â equilibrium that should tell you the number of people you'd expect to vote in

Â any election. Now it's not going to be the same people, right, because the process is

Â going to churn. It's a statistical equilibrium, not a fixed point. But this

Â model, if it's right, if these transition probabilities stay fixed, this would tell

Â us what turnouts should be. Even though it won't tell us who votes. Where else

Â would I use it? Well, we can use it for school enrollment. Same sort of thing, right?

Â [laugh] You can imagine, here's kids who go to school, and here's the ones that

Â don't go to school. And then we could ask, okay at time t+1, how many go, and how many

Â don't go? It could be that of those that go, 90 percent go the next day, and ten

Â percent don't. Of those that don't go, it could be that there's only a 50/50 chance

Â that they come the next day. Well again, if you work through the logic here, you'll

Â get some sort of percentage of people, students, who show up each day, and some

Â percentage that don't, there'll still be a churn, but this model will give you an

Â estimate of what total enrollment should be on a given day. What percentage of

Â students show up. These two applications are very standard applications. They're

Â not unlike the one we looked at, in terms of alert and bored students, and they're

Â not unlike the one with free countries versus dictatorial countries. What I wanna

Â do next is sort of go way outside the box. I wanna just take part of the Markov

Â model. The Markov transition matrix. And I want to think about what that tells us.

Â What it tells us is, if this the state of time t, what are the likelihoods beyond

Â these other states of time t+1? So think about this for a second. There's all

Â sorts of things you could use this framework for, where something happening at

Â time t, and then it transitions into something at time t+1. I'm gonna just talk

Â about three uses of this matrix, three very surprising uses. First one, to

Â identify writers. I mean, [inaudible] use this idea, this transition matrix

Â again, to figure out who wrote a book. So suppose some anonymous person writes a

Â book, and, you're trying to figure out, did this person write it? Did Bob write

Â it? Or you trying to figure out, did you know, Carlos write it? And you can't tell.

Â Well, what you can do is the following: You can figure out transition

Â probabilities. What do I mean? Well, take this book. Take the book written by an

Â anonymous author. And then say, okay, every time this book uses the word "for"--

Â You're loading the whole book in the computer. What percentage of the time does

Â it follow the word "for", with "the record"? What percentage of the time does

Â it follow the word "for" with "example"? And what percentage of the time does it follow

Â the word "for" with "the sake of". And what I'm doing is I'm creating, just, a giant

Â matrix. So if it, in some sense, like, at time t, I'm using "for". And then I'm

Â saying, what's the probability that I follow it with "the record", "example" or "sake"? And I'm

Â putting a .17, .9, .11, so I'm just trying to get a

Â big transition matrix. What you can do is you can take some key words, create these

Â giant transition matrices, and then figure out what does it look like. Does this

Â transition matrix look like Bob's transition matrix, or does it look like

Â Carlos' transition matrix? Now how do I know what Bob's transition matrix is and

Â what Carlos's transition matrix is? Well, that's easy. I just take one of their

Â other books, load it into the computer and figure out what their transition matrix is

Â for the other book. Once I got that in there I can figure out, what, does this

Â look more like Carlos or does this look more like Bob? Now, this actually gets

Â used. And let me tell an interesting story. So this is Arlene Saxenhouse. She's

Â one a of my colleagues at the University of Michigan. And when she was a young

Â graduate student she found, in the library at Yale, this book of, it included four

Â essays by someone who she thought was a young Thomas Hobbes. Book was published in

Â 1620. She thought, 1620, and she thought, oh my gosh, these are essays by Hobbes.

Â And the thing is, she's a young grad student. Who's necessarily going to

Â believe her? How does she prove it? So she couldn't prove it. She had a strong

Â instinct that it was true. Well eventually she found someone who knew how to do this

Â stuff, knew how to do these transition probabilities, and they took the essays

Â and they put in some of Hobbes' other writings, and what they showed was that it

Â seems fairly clear that three of the four essays were actually written by Hobbes, and now

Â those three essays are actually considered part of Hobbes' work. Now, even though

Â appeal, you know, it just sort of felt to her like they were written by Hobbes. That

Â comes down to a matter of opinion. They were having a model, they were having

Â these transition probabilities models. Right? And by able to take that, and take

Â other Hobbes work and this work, you can show statistically that it seems very,

Â very likely that Hobbes wrote the work. So this combination of things. The model

Â alone doesn't do it. The combination of the model, plus her intuition, plus the

Â intuition of others, gives us a common understanding now, at least we think, that

Â Hobbes wrote those particular volumes. It's really cool. Let me give you another

Â example, medical diagnosis. So, if you think about giving someone a treatment for

Â some disease, typically, there's a sequence of reactions to that treatment.

Â Whether it's a drug protocol, or it's an exercise or diet regimen, what you can do

Â is, you can write down transition probabilities. Now, these can be

Â multi-stage. So it could be, for example, that if it's going to be successful, if

Â this treatment is going to be successful, that you go through the following

Â transition: you first feel some pain, then you're slightly depressed, then more pain,

Â but then you get better. Alternatively, if it's not successful, it could be that,

Â initially, you're depressed. Then there's mild pain. Then there's no pain. And then

Â the system fails. Not only the system fails, but you fail, the drug fails, the regimen

Â fails. So what does this mean? This means that, if I give someone the treatment, and

Â then I see this sequence of pain and depression, I can say to them, you know

Â you're feeling pain. You're feeling depression, but guess what, that's

Â consistent with a, a regimen that's gonna be successful. Whereas alternatively

Â someone else could say, well you know, I feel depressed but now I'm definitely not

Â feeling much pain. You can say to them, okay, even though you're not feeling much

Â pain, this probably isn't a good sign. It doesn't look like the treatment is gonna

Â work. So by gathering all sorts of data and past experiences you can use that

Â transition probability, to figure out early on in a treatment protocol, whether it looks like

Â it's working or not working. Another example: Lead up to war. Suppose you got

Â two countries and there's a little bit of tension. So what you get is the following:

Â you get, let's say first you get some political statements on each side. Then

Â that leads to trade embargoes. Then that leads to military buildup. So now you got

Â this sequence of three things. You have these transitions between these three

Â things. You can ask, historically, when I've had those three transitions, what's

Â the likelihood that I've had war, and the likelihood that I haven't had war. And it

Â could be that there's a twenty percent chance of war, and an 80 percent chance of

Â not having war. So, if you're just sort of on the ground watching what going on, you

Â say, oh boy, look at this! First, it was political statements, now there's a trade

Â embargo, now we're seeing military buildups. Looks like things are gonna go to

Â war. If we actually gather a lot of data and, and calculate these transition

Â probability matrices, you could figure out, you know, this actually happens a

Â lot, and only twenty percent of the time does actually lead up to war. So again, we're

Â not using the full power of the Markov model. We're not saying these transition

Â probabilities necessarily stay fixed. We're not worried about solving for the

Â equilibrium. All we're trying to do is just use this matrix to organize the data

Â in such a way that we can think more clearly about what's likely to happen. So

Â that's Markov processes. Markov process is a fixed set of states, fixed transition

Â probabilities. You can get from any one state to any other, and then you get an

Â equilbrium. So that equilibrium doesn't depend on where you start, it doesn't

Â depend on interventions. And it doesn't depend on history in any way. The model is

Â really powerful. And so if you wanna argue history matters. Or if you wanna argue

Â interventions matter. If someone gonna argue that this isn't a transition, that

Â this isn't a Markov process. Or that you've gotta argue that you're changing

Â the transition probabilities. Now that isn't impossible. And in fact, policies that

Â really make a difference, interventions that really make a difference, do change

Â transition probabilities. So what's really cool about this Markov process is,

Â is that it's given us this model. Is it's given us a new lens to look at the world.

Â When we think about it, I want to take this action. It's gonna make things

Â better. We have to be saying, it's changing the transition probabilities, not just

Â that it's changing the state. So if I make my students a little more alert for three

Â seconds by screaming or something, that's not going to change the long running

Â equilibrium of alert and bored students. If I change my teaching style, or if I add more

Â interesting examples in class or something, then it's possible I can change

Â those condition probabilities and end up with more alert students. We've also seen

Â in this last lecture that we don't even need to use the full Markov process model,

Â just the transition probabilities. Just that idea, the matrix of transition

Â probabilities, and we can find out all sorts of interesting things, like who

Â wrote a book? Is there likely to be more? Or is this medical treatment working?

Â So that framework, the transition probability framework and that matrix of

Â transition probabilities is a really powerful tool to keep in your pocket when

Â you confront some sort of dynamic process and you're trying to figure out, what do I think is

Â likely to happen. Thank you.

Â