One issue for fairness is where we have correct results,
but they're misleading or they're unfair in someway.
Here's an obvious thing.
There's a question of how you visualize results or how you present them.
So here's this company and we're looking at their sales year over year.
It looks like fantastic growth curve,
a nice hockey stick with a little upward tick.
Looks really great, isn't it?
Now that's the visual effect.
But if you look at the scale on the left hand side,
the scale along the y axis, we see that really what's happened,
is over a six year time period, sales have gone from 100 to 105.
If you look at this data plotted in a normal, unscaled way,
it might look more like the graph on the right.
What we see is a company that's had a perfectly solid,
decent performance, consistent year over year.
That's certainly no explosive growth at all.
It's got a small, very small in fact,
5% growth over a six year period, nothing to write home about.
Visualization is perhaps most obviously place where
one can have misleading representation of data.
There are many other places that one sees this.
Consider a reputation system,
a travel system where we are looking at user reviews and
using that to choose a hotel to go on a vacation in.
A lot of these systems exists and
they typically give you a rating between 1 and 5.
So we have two hotels here.
Hotel A that gets an average rating of 3.2, and this 3.2,
it turns out comprises a mix of mostly 3's and 4's.
There is another Hotel B which also gets an average of 3.2 but
this 3.2 is a mix of mostly 1's and 5's.
The question is, which hotel would you prefer?
Many reputation websites will just focus on the average and
that's what it'd be ranking hotels by,
that's what you might order your search results by and so on.
And this important difference between the two hotels is
going to be obscured unless you really look into it.
My point is, that something like a hotel B, which people either love or
hate, is a hotel that could either be exactly the perfect hotel for you,
or hotel that you're going to avoid by a mile, depending on whether you're
more like the people who rated a 5 or more like the people who rated a 1.
Calling in a 3.2 is not right because it's either a 5 or
a 1 for you depending on who you are.
And figuring out which of the two you are is not easy without extra work.
And it's certainly not there in the single score that we see.