0:00

Hi. In this lecture we're going to talk about replicator dynamics, and what we

Â want to do is we want to talk about it in the context of learning. And the idea here

Â again is pretty straightforward. What we imagine is, is there are some set of types

Â out there and these types are actions or they're strategies. And each type has a

Â payoff associated with it. That's how well that type is doing. When we look at them,

Â we think boy, these type 1's are doing really well. These type 7's are not doing

Â so well. And then there's also a proportion of each type. So maybe ten

Â percent of people are type one and 30 percent of people are type two, six

Â percent of people are type three and so on. So when you think about how people

Â learn, how do we do, well we've talked about a couple things in this class. One

Â is we realize that we've talked about how people just copy other people, and the

Â reason you might do that is because you might think that they're doing something

Â worthwhile. So in the standing ovation model, we just had people copying what

Â other people do, you don't have conformity models, people just copy what other people

Â do. Well if you copy, what's going to happen is you're gonna copy in proportion

Â what other people are doing. Now another thing you might do is you might hill

Â climb, that was one of our characteristics. If you hill climb, what

Â you're going to do is look and see which actions are paying off well. So when we

Â think about this sort of an environment, there's a whole population of people,

Â existed for proportions and they're getting different payoffs, we want some

Â way of capturing the dynamics of that process. And so what we're going to do is

Â we're going to introduce this model called replicator dynamics that's one way of

Â thinking about how that dynamic unfolds. Now let's again suppose you were rational,

Â So if you're rational you look out there and you see a bunch of types. You see that

Â there's different strategies people are playing: strategy one, strategy two and

Â strategy three. This one has a payoff of five, this one has a payoff of four, this

Â one has a payoff of three. You're just going to say, I'm going to choose strategy

Â one. I look out there and the strategy one people are doing the best. If that's a

Â rational model you could also have a more sociological model, this is a rule based

Â model. You say, 'I'm just going to copy the next person I meet figuring that if

Â they're doing this they must have chosen it for some good reason and I'm going to

Â pick in proportion to other people.' So now if I look at strategy one, strategy

Â two strategy three I can say, twenty percent of people are using strategy one,

Â 70 percent are using strategy two, ten percent using strategy three. I'm more

Â likely to choose strategy two because I'm more likely to bump into someone using

Â strategy two. So there's two ways you could think about how people might choose

Â what to do. One would be to really do a detailed analysis of which actions seem to

Â be paying off the best, be rational and pick that action. Or another thing you

Â could do is you could be sort of just. I was wanting to know, I was just, and I was

Â starting to get photographic magazines and read and look. But I really wanted, wanted

Â to do it so I was doing it first, developing pictures in my. Question is how

Â do we do it. The ideas we're gonna wanna put weight one each possible action and

Â so, and we want that weight to include both the pay off, Which is this thing pi

Â and the proportion, which become probability of i. The one thing we do is

Â we could add those things up. We could say the weight is just the payoff plus the

Â proportion. Another thing we could do is the weight is the product. It's the weight

Â times the proportion. We do either one of these. What we're gonna do is we're gonna

Â use this one. We gonna use the weight is gonna be the probability that the

Â strategy's been used times its payoff. Why? Well, here's why. Suppose that you

Â had something that had a probability equal to zero. So nobody's using this strategy.

Â Well then there would be no way of seeing it, and so it would be very unlik ely that

Â they'd deduce it. But if you think about this model where weight, the weight to an

Â action is equal to the payoff plus the probability, if the payoff is really high,

Â it means the probability is low, people would use it. Well again we think of a

Â population of people, sort of copying rather than learning from other people, if

Â no one is using it. And you couldn't possibly think of it and you couldn't

Â possibly use. So we are not going to have any room for doing anything new. So if the

Â probability is zero, we're just going to assume that there's no one or anybody will

Â ever think of it. So that wipes out this model. We are going to assume the weight

Â is the product than of the path and the strategy and its proportion. So what that

Â means is we get a somewhat complicated procedure for figuring out how many people

Â use the strategy in the next period and this is going to be the replica of the

Â equation. So here's the idea, The probability that you play a strategy, N

Â period T plus one. So the probability of playing strategy on N period T plus one is

Â just the ratio of its weight to the total weight of all the strategies. Cuz remember

Â the weight is just the probably that somebody plays the strategy times its

Â payoff. And on the bottom, We're just summing over the weights of all the

Â different actions or strategies. So the probability you play something in the next

Â period. Probably someone's of that particular type is just gonna be its

Â relative weight. Okay, so let's do an example. We've got three strategies, each

Â one they have payoffs two, four and five, and they exist in proportions a third, a

Â sixth and a half. So what we want to do now is figure out the weight of each of

Â these strategies. So the weight on strategy one is its proportion, which is

Â one-third times its payoff, so that's two-thirds. The weight for strategy two is

Â its proportion, which is 1/6th times its payoff, which is four, which is also

Â two-thirds. And the weight on strategy three is its proportion, which is

Â one-half, times its payoff which is five, whic h is gonna be five halves. So what

Â you can is putting everything over six if we want. So this is four over six. This is

Â four over six. And this is going to be fifteen over six. So if we add up the

Â total weights what we're going to get is 23 over six. So now we want to figure out

Â what's the proportion they're going to be using strategy one in the next period And

Â period two plus one that's just going to be 4/6th over. 23 six which is four over

Â 23. The property in E Strategy two is also four over 23, because it has the same way

Â in the E property Strategy is fifteen over six divided by 23 over six, which is

Â fifteen over 23. You notice if we add that up four plus four plus fifteen gives us 23

Â over 23. So, what we do is we start out with a population, The third strategy one,

Â A sixth strategy two a half strategy three and we end up with Four third twenty-third

Â strategy one, four twenty-third strategy two, and fifteen twenty-third strategy

Â three. That's how replicator dynamics works, it tells us how this population

Â moves over time as a function of the payoffs and the proportions. So here's

Â what we want to do. We want to apply this to games. So here's a very simple game. We

Â can think of this as the shake-bow game where shaking has a higher payoff and we

Â can ask, 'How do the dynamics change?' So what we're going to do is we're going to

Â assume that there's some population of people, some are shakers, some are bowers,

Â and we'll see how that population learns. Alright let's get started. So let's

Â suppose we start out with, one-half shakers and one-half bowers, that's our

Â original population. Now we want to know what's the pay off. So these are our

Â proportions. Well, the pay off, if you're a shaker, half of the time you're going to

Â meet a shaker and half of the time you're going to meet a bower. So if you going to

Â meet a shaker, you're going to get path of one if you meet a bower you're going to

Â get a path of zero. So your payoff is one. If you're a bower, half the time you're

Â gonna need a shaker, you're gonna get a payoff of zero. Half the time you're gonna

Â get a bower and get a payoff of one. So your payoff is gonna be a half. So now we

Â just have to figure out the weight for each of these strategies. So the weight on

Â shaking is just gonna be the proportion, which is one half times the payoff which

Â is one, so that way it's gonna be one half. The way on bowing is the proportion

Â of bowers, which is one half, times the payoff which is one, which is gonna be one

Â fourth. So what we get is, if not to forget them how many shakers of hours

Â [inaudible] in the next period, whats gonna happen is the probability of shakers

Â is gonna equal one half which is the [inaudible] of shakers over the weight of

Â shakers in the balance. So we're gonna get that's just two thirds. In the probability

Â of someone's a power is gonna be one-fourth over one-half plus one-fourth

Â which is one-third. So what we see is we started out with equal number of shakers

Â in powers and now moving towards more shakers but that makes sense because

Â shakers get a higher payoff. Now if we ran this a whole bunch of times and used

Â replica dynamics, And we started out with equal numbers of shakers or bowers,

Â eventually we'd end up with all shakers. So here's an interesting thing, we thought

Â about. Well, how do we model people? We said, well, we should model people as

Â rational. If we thought of rational people, we'd say, well then rational

Â people in this model would choose 2-2. So what replicator dynamics does, it gives us

Â another way to think about what you're gonna get in the game. It says, let's

Â assume a big population of people And let's assume initially that there's equal

Â numbers of each action. So there's equal numbers of shakers, equal numbers of

Â [inaudible]. And in this case sort of, let the population learn according to

Â replicator dynamics, and see what happens. And what we see is in this game.

Â Replicator dynamics would lead us to 2-2. So now if you want you say what's our

Â model of people, you say, 'We have two different models. One model's rational

Â actors, Rational actors are g oing to choose 2-2. Another model is people use

Â this simple rule; this learning rule called [inaudible] dynamics. And if we use

Â this learning rule called [inaudible] dynamics and unless we start out with a

Â whole bunch of hours, we're probably also going to end up With everybody shaking. So

Â that's great, it gives us another motivation for sort of, figuring out why

Â we're going to get the outcomes we're going to get. Well now we can ask the

Â question though, does repetitive dynamics always give us the same thing that we get

Â if we thought about sort of, super smart people playing the game. Well, let's see.

Â Here's another game and this is called the SUV/Compact game. So here's [laugh] how it

Â works. You can either drive an SUV or drive a compact car. If you drive an SUV,

Â your payoff is just gonna be two. Because you just drive your SUV, listen to the

Â radio, it doesn't matter. If you drive a compact car, and you run into someone

Â who's driving an SUV, your payoff is gonna be zero. And I don't mean physically run

Â in, but I mean if you're just driving [inaudible] and you see one, somebody else

Â has an SUV, there's two things going on. One is, you probably can't see around the

Â SUV, so that's bad. And also, you're gonna feel a little bit unsafe, so that's also

Â bad. But, if you're driving a compact car, and the other person is driving a compact

Â car, your path is gonna be three, cause you're both getting better gas mileage,

Â you can see around the other car, you feel safe, everybody wins. So if you're

Â thinking about rational people playing this game, you think, okay, what would you

Â want to do, you might think well look, three three's got the higher payoff, and

Â it's an equilibrium because if we're both driving compacts, then we have no reason

Â to switch. You'd think, that's what you get. Well let's hear what we get from

Â replicator dynamics. So let's start out with again, half the people driving SUVs

Â and half the people driving contracts, compacts. Let's figure out the way. On

Â SUVs. Well, half the people are playing, are driving SU Vs. And your payoff if

Â you're driving an SUV regardless of who you meet is two. So, the weight on SUVs is

Â just gonna be one. What about the weight on compacts? Half the people drive

Â compacts. Now, what's their payoff? If you're driving a compact half the time you

Â meet someone with an SUV. So that gives you a payoff of zero. And half the time

Â you meet someone driving a compact and that give you, so half the time you're

Â gonna get a payoff of three. So that means your total payoff is gonna be three

Â fourths. So the weight on SUV's is one, The weight on compacts is three fourths.

Â So now I wanna ask what's the probability that someone drives an SUV in the next

Â period, That's gonna be one. Over one plus three fourths, which is going to be four

Â sevenths? And if that's what the property is when somebody drives a compact, that's

Â gonna be three fourths over one plus three fourths, which is three sevenths. So, what

Â we see is we're gonna see a drift toward SUV's. And so, what we're gonna get in

Â this game is that people drive SUV's. [inaudible] We put replicate dynamics on

Â this game, we don't get 3,3, as an outcome, we're more likely to get 2,2 as

Â an outcome. And, what's gonna happen is evolution leads us to something that's

Â sub-optimal. So now we've learned something interesting, that these

Â replicator dynamics, this evolution of strategies leads us not to the optimal

Â thing which is 3-3 but leads us to 2-2. There's actually a book written on this

Â that's called high and mighty by Keith Bradshoe, where he talks about, why is it

Â that people drive these big SUVs if it doesn't make sense? Bradshoe makes and

Â argument that it's really through the evolution of choices that have caused us

Â to all be driving SUVs when we'd be collectively be better off if we were

Â driving the compacts. And it's evidenced by this picture here, you can see that in

Â the little car, you're sort of frightened by the big car and you can't see around

Â it, and so as a result, the dynamics, So led us towards big cars, even though we

Â collectively would be better off if we were all driving, Smaller cars. Alright,

Â what have we learned in this lecture? We've learned that one way to think about

Â what people decide to do is to construct a model based on [inaudible] dynamics. And

Â [inaudible] dynamics, they capture two fundamental social processes. One is the

Â fact that people are fairly rational. We copy. We take optimal actions. Another

Â thing that people tend to do, is we tend to copy other people. Cuz if we write down

Â a model where people do both, were we sort of try to do the thing that best but also

Â copy other people or think of combing those, we copy people who are doing well.

Â What we say is that's the way the sort of think about a population of people might

Â go. Now we saw in some cases like the shape [inaudible] game that's going lead

Â us to make the optimum choice. But then we saw other games, like the SUV compact car

Â game that in fact, it didn't let us choose something that is safer the SUV, than not

Â the thing that we choose if we were rational and sat back, and said what's the

Â best choice here. So [inaudible] dynamics really interesting way to model learning

Â and give us sort of surprising insights into some games what is likely to happen,

Â different insights then we get if we assume people were quote UN-quote

Â rational. Alright, Thanks.

Â