Rating Aggregation Challenges

Loading...

From the course by Princeton University

Networks Illustrated: Principles without Calculus

43 ratings

Princeton University

Networks Illustrated: Principles without Calculus

43 ratings

What makes WiFi faster at home than at a coffee shop? How does Google order its search results from the trillions of webpages on the Internet? Why does Verizon charge $15 for every GB of data we use? Is it really true that we are connected in six social steps or less? These are just a few of the many intriguing questions we can ask about the social and technical networks that form integral parts of our daily lives. This course is about exploring the answers, using a language that anyone can understand. We will focus on fundamental principles like “sharing is hard”, “crowds are wise”, and “network of networks” that have guided the design and sustainability of today’s networks, and summarize the theories behind everything from the social connections we make on platforms like Facebook to the technology upon which these websites run. Unlike other networking courses, the mathematics included here are no more complicated than adding and multiplying numbers. While mathematical details are necessary to fully specify the algorithms and systems we investigate, they are not required to understand the main ideas. We use illustrations, analogies, and anecdotes about networks as pedagogical tools in lieu of detailed equations. All the features of this course are available for free. It does not offer a certificate upon completion.

From the lesson

Product Rating on Amazon

The decision of whether or not to purchase something online is often driven by the ratings that previous customers have left for it. In this lesson, we will take a look at Amazon's review system, and the see how "crowds are wise" is another important networking principle.

Amazon & eCommerce4:28

Average Ratings10:21

The Wisdom of Crowds10:03

Rating Aggregation Challenges4:11

Naive Averaging5:57

Bayesian Ranking: Part I5:32

Bayesian Ranking: Part II9:14

Bayesian Ranking in Practice5:41

What does Amazon do? Part I5:00

What does Amazon do? Part II5:29

Meet the Instructors

Christopher Brinton
Lecturer
Electrical Engineering
Mung Chiang
Professor
Electrical Engineering

Let's take a look at three examples that illustrate the

key challenges in deciding when to trust ratings on Amazon.

First, given that different products have a different number of reviews,

as we've said, it's harder to determine which has the best quality.

So, we can refer back to the original example between

the two HD TVs, that we had looked at before.

And as we saw there, one product has four stars

and 95 reviews as opposed to 4.5 stars and eight reviews.

So there's a trade-off between the average review and

the total number of people who have reviewed it.

Which it makes it not exactly clear which of the products is the better one to buy.

And so that trade-off is something that is kind of subjective.

There's no true objective answer to help you, to guide you in that decision.

Second is review variation.

So let's consider two flashlights on Amazon, a Maglite and a Fenix.

So when we look at these they have roughly the same price.

And additionally, so the price isn't shown

here, but the prices are, roughly, the same.

And additionally, we can't say as, last time,

we can't say that there's any difference between

the average customer review, or the number of

reviews because they're, they're, roughly, on the same order.

This is a about 4.5 stars.

This is also about 4.5 stars and 183 reviews versus 260 reviews.

It isn't that large, it not an order of, order of magnitude difference or anything.

So roughly the same number of reviews, roughly the same average customer review.

So, it's not clear which one is the product that you

should buy, but, you know, if you look at, the ratings, what's

interesting is that for the Maglite, you see, there's 169 of

these reviews have five stars, and 20 of them have one star.

So that, that corresponds to 65% being five

stars and, 7.6% being one star reviews.

For the Fenix, we have 62.3% giving five stars, and 2.2% give it one star.

So on the one hand, the number of five star ratings, the fraction of

five star ratings is higher for this flashlight than it is for this one.

But, the fraction of one star ratings is also

higher on this side than it is on this side.

So, again, the question is, does a larger

variation make the average more or less trustworthy?

And so do you go by the fact that

this has, higher fraction of five stars, or that

is has a higher fraction of one stars, in

deciding which, which of these two flashlights you should buy?

Again, the answer to that question is pretty subjective.

Third, we can look at possible recent changes that have occurred.

So, here we're comparing the most recent, the

60 most recent ratings for an iPod Touch.

And the 60 most helpful ratings over all time for this same iPod Touch.

The most helpful ratings has an average of 4.4 right here.

So if we, we take the average of the most, the 60

most helpful ratings over all time, that's going to have an average of 4.4.

But if we look at the most recent ratings

clearly the there's a lot of variation going on.

And average is actually down at 3.6.

So the question is just this trend, this, this recent

increase in variation, this recent decrease in the average value indicate

something that really happened with the product and something that we

need to factor and take into account, displaying the average review?

again, there's no subjective way of determining that.

Whether the Ipod started to have some defect,

or something else is going on, or if it

was just some random fluctuation that is not necessarily

indicative of anything having to do with the product.

The question is, is the most recent, or

the most helpful a better fit for the average?

Again, this as well as the previous

two questions don't really have any clear answers.

Their answers are subjective, and that's

what makes rating aggregation and choosing what

to buy a product kind of difficult on Amazon in any of these three cases.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play