Hi, in this last lecture about Markov processes, what I want to talk about

is exaptation. I want to talk about how we can use the Markov model in context,

and for problems we never would have thought of. And we do this in two ways.

First I'm gonna talk about taking it whole hog, taking the entire process and

modeling other things, just like we modeled the process of states becoming

free or dictatorial. Second, what I wanna do is, I wanna just take part of the Markov

model, its transition probability matrix, and use that to understand some things

that are really kinda surprising and interesting. So let's just remind

ourselves of what a Markov process is. Fixed set of states, and then there's

fixed transition probabilities between those states. Now if it's possible to get

from any state to any other through a sequence of transitions, then remember, we

have this Markov convergence theorem that says the process is going to go to a

unique equilibrium. So history doesn't matter. Initial conditions don't matter.

All that sort of stuff. Let's think about where else we might apply that. So we, one

place right away is voter turnout. So you can think about, there's a set of voters

at time t, and there's a set a of non-voters at time t, and what we can do is

draw a little matrix and say, how many of those are going to vote at time t+1,

and how many of these are not going to vote at time t+1? And it could be that 80

percent of voters at time t vote in t+1, twenty percent don't. It could be of

nonvoters that 40 percent vote and 60 percent don't vote. This would be our

Markov transition matrix and if you apply this, what you'd get is a unique

equilibrium that should tell you the number of people you'd expect to vote in

any election. Now it's not going to be the same people, right, because the process is

going to churn. It's a statistical equilibrium, not a fixed point. But this

model, if it's right, if these transition probabilities stay fixed, this would tell

us what turnouts should be. Even though it won't tell us who votes. Where else

would I use it? Well, we can use it for school enrollment. Same sort of thing, right?

[laugh] You can imagine, here's kids who go to school, and here's the ones that

don't go to school. And then we could ask, okay at time t+1, how many go, and how many

don't go? It could be that of those that go, 90 percent go the next day, and ten

percent don't. Of those that don't go, it could be that there's only a 50/50 chance

that they come the next day. Well again, if you work through the logic here, you'll

get some sort of percentage of people, students, who show up each day, and some

percentage that don't, there'll still be a churn, but this model will give you an

estimate of what total enrollment should be on a given day. What percentage of

students show up. These two applications are very standard applications. They're

not unlike the one we looked at, in terms of alert and bored students, and they're

not unlike the one with free countries versus dictatorial countries. What I wanna

do next is sort of go way outside the box. I wanna just take part of the Markov

model. The Markov transition matrix. And I want to think about what that tells us.

What it tells us is, if this the state of time t, what are the likelihoods beyond

these other states of time t+1? So think about this for a second. There's all

sorts of things you could use this framework for, where something happening at

time t, and then it transitions into something at time t+1. I'm gonna just talk

about three uses of this matrix, three very surprising uses. First one, to

identify writers. I mean, [inaudible] use this idea, this transition matrix

again, to figure out who wrote a book. So suppose some anonymous person writes a

book, and, you're trying to figure out, did this person write it? Did Bob write

it? Or you trying to figure out, did you know, Carlos write it? And you can't tell.

Well, what you can do is the following: You can figure out transition

probabilities. What do I mean? Well, take this book. Take the book written by an

anonymous author. And then say, okay, every time this book uses the word "for"--

You're loading the whole book in the computer. What percentage of the time does

it follow the word "for", with "the record"? What percentage of the time does

it follow the word "for" with "example"? And what percentage of the time does it follow

the word "for" with "the sake of". And what I'm doing is I'm creating, just, a giant

matrix. So if it, in some sense, like, at time t, I'm using "for". And then I'm

saying, what's the probability that I follow it with "the record", "example" or "sake"? And I'm

putting a .17, .9, .11, so I'm just trying to get a

big transition matrix. What you can do is you can take some key words, create these

giant transition matrices, and then figure out what does it look like. Does this

transition matrix look like Bob's transition matrix, or does it look like

Carlos' transition matrix? Now how do I know what Bob's transition matrix is and

what Carlos's transition matrix is? Well, that's easy. I just take one of their

other books, load it into the computer and figure out what their transition matrix is

for the other book. Once I got that in there I can figure out, what, does this

look more like Carlos or does this look more like Bob? Now, this actually gets

used. And let me tell an interesting story. So this is Arlene Saxenhouse. She's

one a of my colleagues at the University of Michigan. And when she was a young

graduate student she found, in the library at Yale, this book of, it included four

essays by someone who she thought was a young Thomas Hobbes. Book was published in

1620. She thought, 1620, and she thought, oh my gosh, these are essays by Hobbes.

And the thing is, she's a young grad student. Who's necessarily going to

believe her? How does she prove it? So she couldn't prove it. She had a strong

instinct that it was true. Well eventually she found someone who knew how to do this

stuff, knew how to do these transition probabilities, and they took the essays

and they put in some of Hobbes' other writings, and what they showed was that it

seems fairly clear that three of the four essays were actually written by Hobbes, and now

those three essays are actually considered part of Hobbes' work. Now, even though

appeal, you know, it just sort of felt to her like they were written by Hobbes. That

comes down to a matter of opinion. They were having a model, they were having

these transition probabilities models. Right? And by able to take that, and take

other Hobbes work and this work, you can show statistically that it seems very,

very likely that Hobbes wrote the work. So this combination of things. The model

alone doesn't do it. The combination of the model, plus her intuition, plus the

intuition of others, gives us a common understanding now, at least we think, that

Hobbes wrote those particular volumes. It's really cool. Let me give you another

example, medical diagnosis. So, if you think about giving someone a treatment for

some disease, typically, there's a sequence of reactions to that treatment.

Whether it's a drug protocol, or it's an exercise or diet regimen, what you can do

is, you can write down transition probabilities. Now, these can be

multi-stage. So it could be, for example, that if it's going to be successful, if

this treatment is going to be successful, that you go through the following

transition: you first feel some pain, then you're slightly depressed, then more pain,

but then you get better. Alternatively, if it's not successful, it could be that,

initially, you're depressed. Then there's mild pain. Then there's no pain. And then

the system fails. Not only the system fails, but you fail, the drug fails, the regimen

fails. So what does this mean? This means that, if I give someone the treatment, and

then I see this sequence of pain and depression, I can say to them, you know

you're feeling pain. You're feeling depression, but guess what, that's

consistent with a, a regimen that's gonna be successful. Whereas alternatively

someone else could say, well you know, I feel depressed but now I'm definitely not

feeling much pain. You can say to them, okay, even though you're not feeling much

pain, this probably isn't a good sign. It doesn't look like the treatment is gonna

work. So by gathering all sorts of data and past experiences you can use that

transition probability, to figure out early on in a treatment protocol, whether it looks like

it's working or not working. Another example: Lead up to war. Suppose you got

two countries and there's a little bit of tension. So what you get is the following:

you get, let's say first you get some political statements on each side. Then

that leads to trade embargoes. Then that leads to military buildup. So now you got

this sequence of three things. You have these transitions between these three

things. You can ask, historically, when I've had those three transitions, what's

the likelihood that I've had war, and the likelihood that I haven't had war. And it

could be that there's a twenty percent chance of war, and an 80 percent chance of

not having war. So, if you're just sort of on the ground watching what going on, you

say, oh boy, look at this! First, it was political statements, now there's a trade

embargo, now we're seeing military buildups. Looks like things are gonna go to

war. If we actually gather a lot of data and, and calculate these transition

probability matrices, you could figure out, you know, this actually happens a

lot, and only twenty percent of the time does actually lead up to war. So again, we're

not using the full power of the Markov model. We're not saying these transition

probabilities necessarily stay fixed. We're not worried about solving for the

equilibrium. All we're trying to do is just use this matrix to organize the data

in such a way that we can think more clearly about what's likely to happen. So

that's Markov processes. Markov process is a fixed set of states, fixed transition

probabilities. You can get from any one state to any other, and then you get an

equilbrium. So that equilibrium doesn't depend on where you start, it doesn't

depend on interventions. And it doesn't depend on history in any way. The model is

really powerful. And so if you wanna argue history matters. Or if you wanna argue

interventions matter. If someone gonna argue that this isn't a transition, that

this isn't a Markov process. Or that you've gotta argue that you're changing

the transition probabilities. Now that isn't impossible. And in fact, policies that

really make a difference, interventions that really make a difference, do change

transition probabilities. So what's really cool about this Markov process is,

is that it's given us this model. Is it's given us a new lens to look at the world.

When we think about it, I want to take this action. It's gonna make things

better. We have to be saying, it's changing the transition probabilities, not just

that it's changing the state. So if I make my students a little more alert for three

seconds by screaming or something, that's not going to change the long running

equilibrium of alert and bored students. If I change my teaching style, or if I add more

interesting examples in class or something, then it's possible I can change

those condition probabilities and end up with more alert students. We've also seen

in this last lecture that we don't even need to use the full Markov process model,

just the transition probabilities. Just that idea, the matrix of transition

probabilities, and we can find out all sorts of interesting things, like who

wrote a book? Is there likely to be more? Or is this medical treatment working?

So that framework, the transition probability framework and that matrix of

transition probabilities is a really powerful tool to keep in your pocket when

you confront some sort of dynamic process and you're trying to figure out, what do I think is

likely to happen. Thank you.