Update: I was unfortunately not aware of Shamus Young's severe criticism of Fallout 3 available here to link in the original piece and I regret that. It dovetails rather nicely with what I've written and it's much better executed than my piece. I strongly recommend anyone...
HomeFeatures GR Showdown: Is Having A Wide Disparity Of Review Scores A Good Thing?
GR Showdown: Is Having A Wide Disparity Of Review Scores A Good Thing?
Posted on Saturday, March 23 @ 01:23:45 Eastern by GR_Staff
GR Showdown pits the Game Revolution staff against each other in a passionate debate on a particular hot-button gaming topic. Our self-imposed rules? There is no middle ground—all must take a side. All debates will have an equal number of representative on both sides: either 1-on-1 or 2-on-2 . And all our arguments must be made in 350 words or fewer; 500 or fewer, if it's 1-on-1. Which side are YOU on?
This Week's Topic: Is Having A Wide Disparity Of Review Scores A Good Thing?
Anthony Severino - YES: This topic hits home for me. Quite frankly, I don’t feel that critics are doing their jobs properly if they’re only using part of the review scale—namely, the seven-to-nine range. Somewhere along the line, most prevalent during this console generation, anything under a seven has been deemed unacceptable by most people’s standards. It’s a shame, to be blunt, because many games scoring a five or a six are still “good” games, especially for someone that’s a fan of that genre or franchise. And when the majority of games attain an eight, these days, how can one find a game that truly stands out? You can’t, unfortunately. And it’s watering down the value of reviews themselves.
Reviewers have a responsibility not only to criticize a game within the text itself, but to be careful with the score that the game is assigned. I hate to say it, but it’s even more important due to Metacritic, because—let’s face it—that’s the real score most people go by these days.
I also find it less believable when a game receives eights across the board from all reviewers. It happens, but it shouldn’t. Tastes differ greatly between people, so why then will one game receive the same score from all reviewers? I guarantee the text reads differently across each review, with different aspects being praised or frowned upon. Yet somehow, they all come to the same conclusion—an eight. I think this has more to do with taking the “easy” way out with scoring, giving a score that’s not too high to where you stand out, but not too low where people hate your guts. In rare cases, admittedly, it just fits the game. But this is, and should be, very rare.
Still, reviewers should—and most do—have a process in which they take personal taste out of the review to give it a proper critique from a more broad perspective. Even then, the final result is still an “opinion” and opinions should vary.
TL;DR - There should be disparity between reviewers, and there needs to be more disparity in the review scale itself.
Daniel Bischoff - NO: This should be pretty obvious to everyone reading and possibly more obvious to the opposition in this debate. There's only one f***ing review you need, and that's GameRevolution's review! 'Nuff said. Oh I still have words left? Let me make it all that much clearer. All the other reviewers out there are wrong and GameRevolution's reviews are right.
Having a wide disparity of review scores does nothing for the consumer; it only serves to benefit the website publishing outlier scores. Oh, the pageviews are down this month? Better trash a great game and reel in those fanboys and their left mouse buttons. Hoping to get in good with a publisher? 5/5 God Tier of Gaming.
A disparity of scores is confusing and accomplishes nothing in moving the medium forward. A score is just a number. Enthusiasts should read reviews from writers they like and trust and share similar tastes with. Your buying decisions shouldn't be dictated by scores A, B, and C. You should feel like you have a personal relationship with your favorite reviewer.
They get you, they know what you like, and they haven't steered you wrong in the past. Trying to reduce scores and bullet points before packing it all into an easily digestible sausage is futile. It's a waste of precious time you could spend gaming. Find an outlet (or two or three) you feel a connection with and build a relationship with the critical writing on hand. Knowing who wrote a review will tell you more about whether you'll enjoy a game more than a number ever will.
Nick Tan - YES: As Anthony inferred, I don't know why reviewers are stuck on the American school scale, where anything less than a 70 is considered a blasphemous pile of fail. But essentially, this entire argument boils down to free speech.
Surely, we hope that reviewers are not simply giving a game an extremely low or high grade for kicks, giggles, and thousands of flame bait hits. Argumentative writing should be a noticeable step over merely stating an opinion and sticking a 0 or 10 on it. Accepting a wide disparity of reviews, however, is necessary every once in a while to remind us that reviewers aren't groupthink robots. We should be allowed to disagree. We're not in a communist state.
As much as consumers may want a general consensus among reviewers, a wide range of scores tends to prompt readers to actually, you know, read more than one review to get a truer sense of where the writers are coming from. Besides, readers can tell if a reviewer is being overly generous or being a hardass prick, as opposed to speaking from the heart. A 4.5 out of 5 for Dragon's Dogma? Why, yes!
There's also the danger of a false consensus, especially in the modern era of embargoes where outlets have to play ball with publishers. By now, you've probably heard how some outlets (which shall remain nameless) have posted their positive reviews early, seemingly creating a consensus of high praise before more negative reviews start piling in after launch.
Even without these timed embargoes, critics follow along: We only need to look at the review for SimCity, which started with high marks before more than several critics were comfortable posting their disastrously low scores after launch. Just check Metacritic and you'll find no low scores before SimCity's March 5th release date. Don't be fooled by consensus.
Alex Osborn - NO: This multifaceted question is a tricky one to tackle. While yes, Anthony, I agree with you that the entire review scale must be used, I disagree slightly with your comments regarding the disparity between reviewers. Of course a review is an opinion, so scores across outlets are bound to differ. But when a game gets a 10 from one outlet and a 2 from another, there's clearly more to the story than just a difference of opinion.
Sadly, we far too often see "critics" either grossly overrate or underrate a game merely to be controversial and generate hits. Not only is this disingenuous to the reader who is looking for an honest take on a game he/she is considering purchasing, but it's also unfair to the team of developers who worked hard to create that game. We writers are a crafty bunch and can formulate an argument around something we may not necessarily even believe ourselves. It's a problem among the gaming press and one that I hope is remedied in the future.
Readers grow accustomed to specific reviewers and the lens through which they see each game. Knowing a particular critic's interests and dislikes gives the reader a better frame of reference when reading a specific review. As long as the reviewer provides an honest opinion of the game, I have no qualms. It's only when the score is altered with the intention of generating hits that I have a problem.
With that said, I would like to reiterate what Daniel mentioned earlier. There's only one review outlet that you should concern yourself with, and that is GameRevolution. We won't ever steer you wrong. I promise.