What makes WiFi faster at home than at a coffee shop? How does Google order its search results from the trillions of webpages on the Internet? Why does Verizon charge $15 for every GB of data we use? Is it really true that we are connected in six social steps or less? These are just a few of the many intriguing questions we can ask about the social and technical networks that form integral parts of our daily lives. This course is about exploring the answers, using a language that anyone can understand. We will focus on fundamental principles like “sharing is hard”, “crowds are wise”, and “network of networks” that have guided the design and sustainability of today’s networks, and summarize the theories behind everything from the social connections we make on platforms like Facebook to the technology upon which these websites run. Unlike other networking courses, the mathematics included here are no more complicated than adding and multiplying numbers. While mathematical details are necessary to fully specify the algorithms and systems we investigate, they are not required to understand the main ideas. We use illustrations, analogies, and anecdotes about networks as pedagogical tools in lieu of detailed equations. All the features of this course are available for free. It does not offer a certificate upon completion.
The Wisdom of Crowds
Loading...
Reviews
4.3 (43 ratings)
- 5 stars23 ratings
- 4 stars13 ratings
- 3 stars6 ratings
- 2 stars1 ratings
From the lesson
Product Rating on Amazon
The decision of whether or not to purchase something online is often driven by the ratings that previous customers have left for it. In this lesson, we will take a look at Amazon's review system, and the see how "crowds are wise" is another important networking principle.
Taught By
Christopher Brinton
Lecturer
Mung Chiang
Professor
So once upon a time back in 1906,
an intriguing contest occurred on farm in Plymouth, UK.
And the idea of this contest was actually to estimate the weight of an ox.
So this ox was placed up on display and the, a bunch of
villagers had to guess the weight of the ox that was put on display.
So the way this worked is that the villagers would come
up, and they would get a good look at the ox and
then they would make a guess, write it down on a sheet
of paper, and maybe put it into some, some bin or something.
They would, they would put it in, submit their guesses in here.
And it was very important also, that they did not look at what
each other, had guessed so that there was no you couldn't copy anyone else.
So you looked at the ox, you formed an opinion of
what of, of what its weight was, and you submitted your guess.
And there were 787 participants.
So 787 villagers who participated in this experiment.
So Sir Francis Galton analyzed these results, and he
wanted to see what he could come up with.
And that one person guessed the true weight of the ox which was 1,198 pounds.
So, so no one guessed the true weight.
But the average of the guesses was 1,197 pounds
which is an error of only less than 0.1%.
So, very, very small error in this guess.
So you're probably wondering, well, how could that be, right?
Because nobody was anywhere near the true guess.
They were all over the place.
And some people guessed way too high.
Some people guessed way too low.
But when we average them, we get something really close, to what the true value was.
So as it turns out there were several factors
at work here, that played into favor of this
whole guessing game and averaging of the guesses, actually
coming out to be something, very meaningful and very accurate.
The first one is that the task is relatively easy, right?
So you just, you look at the ox and you guess the weight and you put it in, right?
It's a very objective task.
The second is that the estimates were independent and unbiased.
So independent was the, basically the idea the way of that is made,
is that nobody was able to look at any, what anybody else submitted.
So everybody does their task entirely independently, and it's not, so what
you guess is not going to be dependent upon what anybody else guesses.
Then unbiased means that there was no systematic tendency for anyone
to guess, or for everyone to guess higher or lower, right?
And the way to create that in this situation would be something like,
if somebody wanted the true, the people to guess too high or too low
and they added some, some decoration on top of the ox and made it
look like it was heavier than it was, then everybody might guess too high.
So that's the idea.
So, if we draw this out here, let's just put everything on a number line.
And we say that the true value whatever this is
in general, this guessing game is right here at the center.
And say this is something like 100.
So the idea is that if I submit a guess maybe I'll say 90, for whatever it is.
Maybe then you submit a guess and you say 120, or maybe you're smarter than I am.
So I guess 120 which is 20 off and you guessed 90 which is only ten off.
Someone else comes in makes a guess here.
Someone else makes a guess here.
Someone makes a guess, guess more and more.
The idea is that even though they're far
off, they're going to concentrate around this center point.
So the concentration's going to be around here, and when we
average them we're going to get something close to that center.
So if they were biased just so you understand what bias is, right?
This is the true mean.
So there would have to be some systematic tendency to
guess higher than this, so that everybody would guess higher.
And, if it was biased then that would mean
that this distribution was actually centered up around here.
So this would be a bias.
So everybody, instead of, you guessing 120,
or instead of me guessing 120, I would
have guessed, 180 or something and, you
would've guessed, the true value was being 140.
So, there, there aren't values below and above.
There's only values above.
And similarly if there was a bias below, there would only
be values below or they would concentrate around the center, center below.
The third is that there were enough people participating here.
So 787 participants is enough to get a,
a good average in this case for this experiment.
And as we'll see the number of people that need to
participate is going to depend upon really how difficult the task is.
And also how independent and how unbiased the estimates are.
So returning now to Amazon, our hope is that when we average customer
ratings for a product, the result is going to get close to the right rating.
But can we even say that such a true even exists?
And doesn't this depend at least somewhat on the specific customer?
So, let's look at the factors that we just identified as
being important in Galton's experiment in the context of Amazon reviews.
First is the definition of task.
So, guessing numbers is a pretty easy task to do.
Guessing the weight of an ox is a straight-forward task.
Reviewing a product on Amazon, answering a rating is a little more difficult.
The second is the independence of the reviews.
Success in opinion aggregation stems not from having many smart
individuals in a crowd who are likely to guess correctly.
Rather, it's from the independent each individual's view
from the rest, as we talked about before.
My review isn't dependent upon yours.
I form my own opinion.
So the question is, are Amazon reviews independent of each other?
And they kind of are, not, not entirely though.
Because you could go on and look at the previous reviews
of a product and read through them before you submit your own.
But given that you've tried out the
product on yourself, you're probably going to form
somewhat of you own opinion at least and submit somewhat of an independent review.
Third one is the review population.
So Galton's experiment would not have worked as well as
we said if there had not been 787, 800 people participating.
And in general, the harder the task is and the less independent the
reviews are, the less unbiased and the
less independent they are from one another.
The more people we're going to need in order to get a good
estimator from the average, or from the collective guess at the population.
And so, those factors are all going to come into play, here.
So, wisdom of crowds we can actually
summarize this, what we've been saying, mathematically.
The idea, intuitively again, is that when we take the opinion of a massive crowd
it's going to be greater than the opinion of any one single person.
So, what it says is that the error in the average, so if we
take a number of guesses, so for instance, this is for these five people here.
We take their guesses, and we take the average of their guesses.
The error that that average is going to have, roughly speaking, is
going to be the error that each guest has, you know, assuming that
they each have the same, not, not the same error, but
the same distribution in their error divided by the number of people.
So it's going to lower by the number of people that are in that estimate.
So if there's five people, it's going to be five
lower than the error that each guess has on average.
So when we take the average, we're going to start to see less and less guess.
And the reason that we use an approximation symbol here is
because naturally not every guess is going to have the same error.
And in fact, if they all had the same exact error this
wouldn't help at all because then they would all be biased up.
That's would actually be a bias condition there and it wouldn't help at all then.
What we need, we need all these errors to be able
to cancel each other out, but if we take the expected
value of the error we say statistically, divided by the number
of people, it's going to have the expected error in the average.
So, very important thing though, is that this is
only going to hold if the guesses are independent and unbiased.
And the reason that, that needs to hold this
because these errors intuitively need to cancel each other out.
Again, if my, if the true value is over
here, right, we can't all be guessing up here, right?
Or else, when we average out we're going to be nowhere close to the mean.
We have to some higher, some lower than the average, and so forth.
And we have to have them not being dependent upon one another either.
So, independence and unbias are absolutely essential
for the wisdom of crowds to hold up.
So just as the example of application of this equation
suppose we have one person, right, and his guess is three.
So the average here, is going to be three divided by
one, which is equal to three, because there's only one person.
Sorry, three.
And suppose that the error, is one.
So we expect him to be too high by one.
Now, suppose we instead have five people, and they
submit guesses of three, two, three, four and four.
So there's five people here, down the line.
And they submit their guesses.
So if we take the average of this, we're going to have
three plus two plus three plus four plus four, divided by five.
And that comes out to be 3.2.
And the error here, the important part, is the error of
one person, divided by the total number of people, which was five.
So, the error, when we have five people, produces from one in this case to .2.
So now, suppose we have a large number of reviewers for a product on Amazon.
Does this discussion imply that the average rating is
going to be close to the truth that we want?
Well, not necessarily, because as we
pointed out, they're many complications, right?
The task is not exactly easy.
There's not exact independence assumption, and their
view population may not be large enough.
And so let's explore a few of these challenges next.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
