Explore
For Enterprise
Join for Free
Log In

Arts and Humanities
Business
Computer Science
Data Science
Information Technology
Health
Math and Logic
Personal Development
Physical Science and Engineering
Social Sciences
Language Learning
Degrees
Certificates

Explore all of Coursera

Browse
Search
For Enterprise
Log In
Join for Free

The Wisdom of Crowds

Loading...

Networks Illustrated: Principles without Calculus

Princeton University

4.4 (46 ratings) | 11K Students Enrolled

What makes WiFi faster at home than at a coffee shop? How does Google order its search results from the trillions of webpages on the Internet? Why does Verizon charge $15 for every GB of data we use? Is it really true that we are connected in six social steps or less? These are just a few of the many intriguing questions we can ask about the social and technical networks that form integral parts of our daily lives. This course is about exploring the answers, using a language that anyone can understand. We will focus on fundamental principles like “sharing is hard”, “crowds are wise”, and “network of networks” that have guided the design and sustainability of today’s networks, and summarize the theories behind everything from the social connections we make on platforms like Facebook to the technology upon which these websites run. Unlike other networking courses, the mathematics included here are no more complicated than adding and multiplying numbers. While mathematical details are necessary to fully specify the algorithms and systems we investigate, they are not required to understand the main ideas. We use illustrations, analogies, and anecdotes about networks as pedagogical tools in lieu of detailed equations. All the features of this course are available for free. It does not offer a certificate upon completion.

Reviews

4.4 (46 ratings)

5 stars
26 ratings
4 stars
13 ratings
3 stars
6 ratings
2 stars
1 ratings

From the lesson

Product Rating on Amazon

The decision of whether or not to purchase something online is often driven by the ratings that previous customers have left for it. In this lesson, we will take a look at Amazon's review system, and the see how "crowds are wise" is another important networking principle.

Amazon & eCommerce4:28

Average Ratings10:21

The Wisdom of Crowds10:03

Rating Aggregation Challenges4:11

Naive Averaging5:57

Bayesian Ranking: Part I5:32

Bayesian Ranking: Part II9:14

Bayesian Ranking in Practice5:41

What does Amazon do? Part I5:00

What does Amazon do? Part II5:29

Taught By

Christopher Brinton
Lecturer
Mung Chiang
Professor

Transcript

So once upon a time back in 1906,

an intriguing contest occurred on farm in Plymouth, UK.

And the idea of this contest was actually to estimate the weight of an ox.

So this ox was placed up on display and the, a bunch of

villagers had to guess the weight of the ox that was put on display.

So the way this worked is that the villagers would come

up, and they would get a good look at the ox and

then they would make a guess, write it down on a sheet

of paper, and maybe put it into some, some bin or something.

They would, they would put it in, submit their guesses in here.

And it was very important also, that they did not look at what

each other, had guessed so that there was no you couldn't copy anyone else.

So you looked at the ox, you formed an opinion of

what of, of what its weight was, and you submitted your guess.

And there were 787 participants.

So 787 villagers who participated in this experiment.

So Sir Francis Galton analyzed these results, and he

wanted to see what he could come up with.

And that one person guessed the true weight of the ox which was 1,198 pounds.

So, so no one guessed the true weight.

But the average of the guesses was 1,197 pounds

which is an error of only less than 0.1%.

So, very, very small error in this guess.

So you're probably wondering, well, how could that be, right?

Because nobody was anywhere near the true guess.

They were all over the place.

And some people guessed way too high.

Some people guessed way too low.

But when we average them, we get something really close, to what the true value was.

So as it turns out there were several factors

at work here, that played into favor of this

whole guessing game and averaging of the guesses, actually

coming out to be something, very meaningful and very accurate.

The first one is that the task is relatively easy, right?

So you just, you look at the ox and you guess the weight and you put it in, right?

It's a very objective task.

The second is that the estimates were independent and unbiased.

So independent was the, basically the idea the way of that is made,

is that nobody was able to look at any, what anybody else submitted.

So everybody does their task entirely independently, and it's not, so what

you guess is not going to be dependent upon what anybody else guesses.

Then unbiased means that there was no systematic tendency for anyone

to guess, or for everyone to guess higher or lower, right?

And the way to create that in this situation would be something like,

if somebody wanted the true, the people to guess too high or too low

and they added some, some decoration on top of the ox and made it

look like it was heavier than it was, then everybody might guess too high.

So that's the idea.

So, if we draw this out here, let's just put everything on a number line.

And we say that the true value whatever this is

in general, this guessing game is right here at the center.

And say this is something like 100.

So the idea is that if I submit a guess maybe I'll say 90, for whatever it is.

Maybe then you submit a guess and you say 120, or maybe you're smarter than I am.

So I guess 120 which is 20 off and you guessed 90 which is only ten off.

Someone else comes in makes a guess here.

Someone else makes a guess here.

Someone makes a guess, guess more and more.

The idea is that even though they're far

off, they're going to concentrate around this center point.

So the concentration's going to be around here, and when we

average them we're going to get something close to that center.

So if they were biased just so you understand what bias is, right?

This is the true mean.

So there would have to be some systematic tendency to

guess higher than this, so that everybody would guess higher.

And, if it was biased then that would mean

that this distribution was actually centered up around here.

So this would be a bias.

So everybody, instead of, you guessing 120,

or instead of me guessing 120, I would

have guessed, 180 or something and, you

would've guessed, the true value was being 140.

So, there, there aren't values below and above.

There's only values above.

And similarly if there was a bias below, there would only

be values below or they would concentrate around the center, center below.

The third is that there were enough people participating here.

So 787 participants is enough to get a,

a good average in this case for this experiment.

And as we'll see the number of people that need to

participate is going to depend upon really how difficult the task is.

And also how independent and how unbiased the estimates are.

So returning now to Amazon, our hope is that when we average customer

ratings for a product, the result is going to get close to the right rating.

But can we even say that such a true even exists?

And doesn't this depend at least somewhat on the specific customer?

So, let's look at the factors that we just identified as

being important in Galton's experiment in the context of Amazon reviews.

First is the definition of task.

So, guessing numbers is a pretty easy task to do.

Guessing the weight of an ox is a straight-forward task.

Reviewing a product on Amazon, answering a rating is a little more difficult.

The second is the independence of the reviews.

Success in opinion aggregation stems not from having many smart

individuals in a crowd who are likely to guess correctly.

Rather, it's from the independent each individual's view

from the rest, as we talked about before.

My review isn't dependent upon yours.

I form my own opinion.

So the question is, are Amazon reviews independent of each other?

And they kind of are, not, not entirely though.

Because you could go on and look at the previous reviews

of a product and read through them before you submit your own.

But given that you've tried out the

product on yourself, you're probably going to form

somewhat of you own opinion at least and submit somewhat of an independent review.

Third one is the review population.

So Galton's experiment would not have worked as well as

we said if there had not been 787, 800 people participating.

And in general, the harder the task is and the less independent the

reviews are, the less unbiased and the

less independent they are from one another.

The more people we're going to need in order to get a good

estimator from the average, or from the collective guess at the population.

And so, those factors are all going to come into play, here.

So, wisdom of crowds we can actually

summarize this, what we've been saying, mathematically.

The idea, intuitively again, is that when we take the opinion of a massive crowd

it's going to be greater than the opinion of any one single person.

So, what it says is that the error in the average, so if we

take a number of guesses, so for instance, this is for these five people here.

We take their guesses, and we take the average of their guesses.

The error that that average is going to have, roughly speaking, is

going to be the error that each guest has, you know, assuming that

they each have the same, not, not the same error, but

the same distribution in their error divided by the number of people.

So it's going to lower by the number of people that are in that estimate.

So if there's five people, it's going to be five

lower than the error that each guess has on average.

So when we take the average, we're going to start to see less and less guess.

And the reason that we use an approximation symbol here is

because naturally not every guess is going to have the same error.

And in fact, if they all had the same exact error this

wouldn't help at all because then they would all be biased up.

That's would actually be a bias condition there and it wouldn't help at all then.

What we need, we need all these errors to be able

to cancel each other out, but if we take the expected

value of the error we say statistically, divided by the number

of people, it's going to have the expected error in the average.

So, very important thing though, is that this is

only going to hold if the guesses are independent and unbiased.

And the reason that, that needs to hold this

because these errors intuitively need to cancel each other out.

Again, if my, if the true value is over

here, right, we can't all be guessing up here, right?

Or else, when we average out we're going to be nowhere close to the mean.

We have to some higher, some lower than the average, and so forth.

And we have to have them not being dependent upon one another either.

So, independence and unbias are absolutely essential

for the wisdom of crowds to hold up.

So just as the example of application of this equation

suppose we have one person, right, and his guess is three.

So the average here, is going to be three divided by

one, which is equal to three, because there's only one person.

Sorry, three.

And suppose that the error, is one.

So we expect him to be too high by one.

Now, suppose we instead have five people, and they

submit guesses of three, two, three, four and four.

So there's five people here, down the line.

And they submit their guesses.

So if we take the average of this, we're going to have

three plus two plus three plus four plus four, divided by five.

And that comes out to be 3.2.

And the error here, the important part, is the error of

one person, divided by the total number of people, which was five.

So, the error, when we have five people, produces from one in this case to .2.

So now, suppose we have a large number of reviewers for a product on Amazon.

Does this discussion imply that the average rating is

going to be close to the truth that we want?

Well, not necessarily, because as we

pointed out, they're many complications, right?

The task is not exactly easy.

There's not exact independence assumption, and their

view population may not be large enough.

And so let's explore a few of these challenges next.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play

Coursera
About
Leadership
Careers
Catalog
Certificates
MasterTrack™ Certificates
Degrees
For Enterprise
For Government

Community
Learners
Partners
Developers
Beta Testers
Translators

Connect
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Tech Blog

More
Terms
Privacy
Help
Accessibility
Press
Contact
Directory
Affiliates