Something's Wrong With Reviews Today (And Reviewers)
Posted on Thursday, October 27 @ 14:14:40 Eastern by Anthony Severino
Yesterday's fiasco involving an IGN editor and a PlayStation Network developer got me thinking... video game reviews are a disaster.
I'm in no way, shape, or form defending this particular editor - he wasn't completely truthful and he got caught - but there's a lot more at play here than meets the eye.
Part of the problem is that there is so much competition in the gaming space, that every site must fight tooth and nail to get their scraps of traffic. And it just so happens that nothing brings in traffic faster than being "first" on the web. Being first means that every single person who searches for this particular topic sees your publication at the top of Google's search results. While this is possible to be accomplished organically, people even pay for this via Google Adwords.
How many times have you Googled something, and didn't go past the first couple results, let alone the first page of search query results?
I admit, I have no examples of such instances, but corners can be, and will be cut in order to be the first publication with a review published. I just saw this recently with Uncharted 3: Drake's Deception, where a reviewer clearly didn't play the single-player campaign and instead reviewed the multiplayer, and talked about bits and pieces pertaining to the storyline - stuff that's already been publicly revealed via trailers, etc.
With this IGN review, I can't say if this is indeed the case. But blazing through a game on easy in order to finish quickly isn't getting the full experience of the game. That much I know.
I've been an Editor-In-Chief for 4 years running - I know a lot about deadlines. Sometimes they're unreasonable, but are a necessary evil. I've at times told Daniel not to post a single piece of news content for the entire day, in order to play though a game in its entirety, to ensure we get a review out in a timely manner. More pressure is applied if the game warrants it.
Just like any job, if you're not completing projects by set deadlines, you risk losing your job. Maybe not the first time, but if it's frequent, or if you're new, it's certainly possible. And when you've got as large of a staff as IGN does, there simply has to be less leniency in policy. Otherwise, things will get chaotic.
Late review copies
This is completely out of any journalist's hands. Some titles arrive late - so very late - forcing journalists to sacrifice sleep, relationships, and nearly everything else in order to get a review out on time. It also can facilitate the need to rush or skip certain portions of a game. Maybe you don't complete all of the side-quests, or maybe you don't compete in as many of the multiplayer modes as you'd like - again, sacrifices must be made.
Take Batman: Arkham City for example - Warner Bros. screwed us. They sent us our review copy the day the game came out. It arrived at our office mid-day on release day, and long after the IGN's of the world had their reviews up. I wasn't waiting around, so I had Daniel head over the midnight launch at a local retailer and pick it up Monday night before our review copy even arrived. That way he could get started on the review right away. Daniel's review was posted only two days later, and I'd stake my reputation on his thoroughness. I know, because he literally put 30 hours into playing it, in a matter of 48 hours. How much time did that really give him for anything else but playing and writing? It sucks, it's not fair, but it's something we deal with often.
It's not always the case - sometimes we get review copies very early, which give us ample time to produce something and have it ready by embargo or release date. Other times they don't send a copy at all (which usually means the game sucks and they're avoiding poor review scores pre-launch).
Let's face it - sometimes people aren't cut out for the jobs they are in. The game industry is no different. For every Jeff Gerstmann or Geoff Keighley, there's twenty wanna-bes willing to do whatever it takes to reap the benefits of being a games journalist. They'll cut corners, take bribes, and who knows what else? I'm not saying that this particular IGN reviewer is a hack--this is the first I've heard of him--I'm just saying that not everyone puts forth the same effort, or can produce in the same quality and capacity.
The good thing is, the hacks can't cut it for long, and eventually fade away. Meanwhile the real, respected journalists go on to do great things.
Four score and seven years ago
This is only somewhat related to the topic at hand, and one that calls for a feature of its own and days or months of research to ever really explain the entire picture - but, review scores are completely fucked. This entire industry's review system is completely fucked...
This reviewer explains on his blog that this developer was "upset" over a poor review score of his game. The score was a 6.5. To me, that's above average. To video game consumers, forum goers, fanboys, and internet trolls alike, this means the game is shit. Your average video game outlet uses a 10 scale, but anything graded below a 7 is considered blasphemy by the general public.
It's not just the public that's at fault - I once opened a copy of PlayStation: The Official Magazine (this was only months ago), and I gazed in amazement at the reviews section. Every single game--except for one lonely 7--scored an 8. I even peered ahead in the issue to see their Blu-ray scores, only to find three movies scoring... a fucking 8. I remember turning to my wife and doing this insane, sinister laugh. I said "they mean to tell me that every single one of the 20 or so games they reviewed in this issue are of the EXACT same quality"?
Look at metacritic, anything that scores below 70 or so is considered to be a flop. And this indsutry lives and dies by metacritic. People's bonus for a game they worked on for three years depends hinges on a metascore that meets the demands of a CEO and shareholders.
This situation sucks. It was blown out of proportion. The guy played through the game on casual mode, and his review reflected that. When called on it, he removed the blurb that proved the developer's claim. Although the reviewer came forward to explain himself, he'll never again be fully trusted - not just by readers but by publishers and developers. And for what reason? He probably played through the game on casual to meet a deadline, so that IGN's review could be first, on a game that was already released, which forced him to take the easy way out.
To say it's not his fault would be wrong, but to say it's all his fault would be just as wrong.
For another inside look at the review process, be sure to check out our 12 Steps to Video Game Reviews.
FOLLOW ME ON TWITTER. YOU KNOW YOU WANT TO.