More Reviews
REVIEWS Hatoful Boyfriend Review
Before this review, there seemed to be an absence of a certain ornithological piece; a headline regarding mass awareness of a certain avian variety...

Murasaki Baby Review
When a little girl with an upside-down face finds a balloon, you better protect it from monsters!
More Previews
PREVIEWS Halo: The Master Chief Collectio Preview
Microsoft and 343 Industries want to bring the entire Halo saga to Xbox One and this collection does exactly that with new graphics for Halo 2.
Release Dates
NEW RELEASES Persona 4 Arena Ultimax
Release date: 09/30/14

Alien: Isolation
Release date: 10/07/14

Borderlands: The Pre-Sequel
Release date: 10/14/14

The Evil Within
Release date: 10/14/14


LATEST FEATURES Assassin's Creed Unity Interview: Ubisoft Talks Multplayer, Next-Gen Development, More
Ubisoft's first "truly" next-gen entry in the Assassin's Creed franchise takes the fight to France. Here's what you can expect.

PlayStation Download September 2014 - Updating Each Week
Sony's platforms always get plenty of new digital software and we'll bring you the list each week with the rest.
MOST POPULAR FEATURES The Updating List of PAX Indies
We're heading to PAX Prime! Are you looking to check out a few unique indie games while you're there? UPDATED: Dragon Fin Soup, Dungeon of the Endless,

LEADERBOARD
Read More Member Blogs
FEATURED VOXPOP shandog137
A Letter to the Big “N"
By shandog137
Posted on 09/12/14
I have and will continue to have a place in my heart for Nintendo. In fact, my first console was a Super Nintendo. The video game market has changed drastically since the early '90s and it seems like what once was platinum is more so along the lines of silver now. Nintendo has always been...

Amazon Customer Reviews Are Extremely Inflated

Posted on Thursday, November 21 @ 13:42:17 Eastern by
In response to my review for Knack, in which I gave it a low score of 2 stars, comments I've read on GameRevolution and elsewhere have expressed that people should take low reviews for Knack with a grain of salt because user ratings and customer reviews, particularly those on Amazon, are positive. Indeed, I checked and, at the time of this writing, it has a 4-star rating out of 5. That seems like a far cry from the game's Metascore of 56.

So what the hell is going on here?! Are reviewers just some group of jaded cynics who enjoy being contrarian for the sake of it? Are we just out of touch with "the people"? Or is there something deeper going on with professional review scores and non-professional customer reviews on Amazon?

Well, this isn't the first time that there's been a disparity. Here's a chart comparing the Metascores and the Amazon customer review scores (scaled to 100 to match the Metascore) of twenty of the lower-scoring titles in the past three years:



Whatever the case may be, some kind of strange fidgeting seems to be going on with Amazon reviews. No matter how terrible professional reviewers seem to give a game, Amazon reviews don't seem to drop lower than 2.8 stars (or a 56 when scaled). Of course, there are more than a few exceptions to the rule, as this is just a small sampling, but something weird is going on.

Some sites have already warned that up to 30% of user-generated online reviews are fake (some more conservative estimates have this at around 10-15%). There's certainly a vested interest for publishers and developers to create customer reviews that are positive for their products to generate and maintain sales, though I believe that most of the larger companies would avoid this as it's, well, fraudulent. Now I'm sure this happens in small numbers, but that doesn't fully account for the stark differences in some of the figures in the chart.

Indeed, a large part of the discrepancy comes from the mathematics. Professional review sites generally have a scale that goes from 0 to 100, or 0 to 5 stars, but Amazon as well as other retailer sites have a scale that only goes from 1 to 5 stars. This essentially drops the lowest part of the grading scale, effectively making the scale from 20 to 100 and increasing the average score by default. That's one of the reasons that you see Amazon customer review scores hovering around the average 3-star rating.



Then you have post-purchase rationalization, also known as Buyer's Stockholm Syndrome, where the customer has a bias for the choices they make; that is, the purchases they make. Someone who has already purchased a product will tend to overlook its faults, and given that most games cost $60, which most consumers consider to be somewhat expensive, gamers are more resistant on attacks upon their judgment. It's a psychological effect of commitment.

Users are also less bound to the strict scale that reviewers have and have no qualms giving games the lowest score or the highest score. So any games that receive even a small number of 5-star ratings will have a much lower probability of receiving less than an overall 3-star rating. (It takes two 1-star reviews to knock one 5-star rating to an average of 2-stars.) Combine this with the other factors and it makes this almost a certainty.

In other words, Amazon customer ratings are inflated. That doesn't mean they shouldn't be trusted insomuch as you should expect them to be much higher on average, especially for games that critics have ravaged. Certainly, it's up to you to decide which side of the fence has more piles of salt. Just know that there's plenty to go around...
Tags:   Amazon, Review
FOLLOW ME ON TWITTER. YOU KNOW YOU WANT TO.

comments powered by Disqus




More On GameRevolution