More Reviews
REVIEWS Resident Evil: Revelations Review
While 3DS gamers have been enjoying the franchise's best game in years for some time now, does the experience translate for Resident Evil fans on console?

Donkey Kong Country Returns 3D Review
Gamers have gone bananas for Nintendo's 3DS, but can this port of Retro Studios' 2010 Wii game make the jump to your portable?
More Previews
PREVIEWS The Last of Us Preview
With Naughty Dog releasing a new IP in just a few short weeks, we got hands-on one more time. But don't worry: This is a spoiler-free preview.
Release Dates
NEW RELEASES GRiD 2
Release date: 05/28/13

Fuse
Release date: 05/28/13

Remember Me
Release date: 06/04/13

The Last of Us
Release date: 06/14/13


LATEST FEATURES Being A Console Is Actually Xbox One's Worst Asset
Microsoft's newest console has lots of different features, but video games might hold the device back from the software giant's true intentions.

Everything I Learned About Call of Duty: Ghosts Last Week
I wasn't allowed to talk about the new Infinity Ward game last week when I met with Activision, and I don't have much to say now that Xbox One spilled the beans.
 
Coming Soon

LEADERBOARD
Read More Member Blogs
FEATURED VOXPOP Bras
On the future of some gamers
By Bras
Posted on 05/22/13
Before Microsoft and Sony do something regarding their future in the video game business, I wanted to write, and I've wanted it for a long time now, but other things kept getting in my way, and fearing that tomorrow might be too late, today will have to do.  ...

Something's Wrong With Reviews Today (And Reviewers)

Posted on Thursday, October 27 @ 14:14:40 Eastern by


Yesterday's fiasco involving an IGN editor and a PlayStation Network developer got me thinking... video game reviews are a disaster.

I'm in no way, shape, or form defending this particular editor - he wasn't completely truthful and he got caught - but there's a lot more at play here than meets the eye.

FIRST!


Part of the problem is that there is so much competition in the gaming space, that every site must fight tooth and nail to get their scraps of traffic. And it just so happens that nothing brings in traffic faster than being "first" on the web. Being first means that every single person who searches for this particular topic sees your publication at the top of Google's search results. While this is possible to be accomplished organically, people even pay for this via Google Adwords.

How many times have you Googled something, and didn't go past the first couple results, let alone the first page of search query results?



I admit, I have no examples of such instances, but corners can be, and will be cut in order to be the first publication with a review published. I just saw this recently with Uncharted 3: Drake's Deception, where a reviewer clearly didn't play the single-player campaign and instead reviewed the multiplayer, and talked about bits and pieces pertaining to the storyline - stuff that's already been publicly revealed via trailers, etc.

With this IGN review, I can't say if this is indeed the case. But blazing through a game on easy in order to finish quickly isn't getting the full experience of the game. That much I know.

Deadlines


I've been an Editor-In-Chief for 4 years running - I know a lot about deadlines. Sometimes they're unreasonable, but are a necessary evil. I've at times told Daniel not to post a single piece of news content for the entire day, in order to play though a game in its entirety, to ensure we get a review out in a timely manner. More pressure is applied if the game warrants it.

Just like any job, if you're not completing projects by set deadlines, you risk losing your job. Maybe not the first time, but if it's frequent, or if you're new, it's certainly possible. And when you've got as large of a staff as IGN does, there simply has to be less leniency in policy. Otherwise, things will get chaotic.



Late review copies


This is completely out of any journalist's hands. Some titles arrive late - so very late - forcing journalists to sacrifice sleep, relationships, and nearly everything else in order to get a review out on time. It also can facilitate the need to rush or skip certain portions of a game. Maybe you don't complete all of the side-quests, or maybe you don't compete in as many of the multiplayer modes as you'd like - again, sacrifices must be made.

Take Batman: Arkham City for example - Warner Bros. screwed us. They sent us our review copy the day the game came out. It arrived at our office mid-day on release day, and long after the IGN's of the world had their reviews up. I wasn't waiting around, so I had Daniel head over the midnight launch at a local retailer and pick it up Monday night before our review copy even arrived. That way he could get started on the review right away. Daniel's review was posted only two days later, and I'd stake my reputation on his thoroughness. I know, because he literally put 30 hours into playing it, in a matter of 48 hours. How much time did that really give him for anything else but playing and writing? It sucks, it's not fair, but it's something we deal with often.

It's not always the case - sometimes we get review copies very early, which give us ample time to produce something and have it ready by embargo or release date. Other times they don't send a copy at all (which usually means the game sucks and they're avoiding poor review scores pre-launch).

Hacks


Let's face it - sometimes people aren't cut out for the jobs they are in. The game industry is no different. For every Jeff Gerstmann or Geoff Keighley, there's twenty wanna-bes willing to do whatever it takes to reap the benefits of being a games journalist. They'll cut corners, take bribes, and who knows what else? I'm not saying that this particular IGN reviewer is a hack--this is the first I've heard of him--I'm just saying that not everyone puts forth the same effort, or can produce in the same quality and capacity.

The good thing is, the hacks can't cut it for long, and eventually fade away. Meanwhile the real, respected journalists go on to do great things.



Four score and seven years ago


This is only somewhat related to the topic at hand, and one that calls for a feature of its own and days or months of research to ever really explain the entire picture - but, review scores are completely fucked. This entire industry's review system is completely fucked...

This reviewer explains on his blog that this developer was "upset" over a poor review score of his game. The score was a 6.5. To me, that's above average. To video game consumers, forum goers, fanboys, and internet trolls alike, this means the game is shit. Your average video game outlet uses a 10 scale, but anything graded below a 7 is considered blasphemy by the general public.

It's not just the public that's at fault - I once opened a copy of PlayStation: The Official Magazine (this was only months ago), and I gazed in amazement at the reviews section. Every single game--except for one lonely 7--scored an 8. I even peered ahead in the issue to see their Blu-ray scores, only to find three movies scoring... a fucking 8. I remember turning to my wife and doing this insane, sinister laugh. I said "they mean to tell me that every single one of the 20 or so games they reviewed in this issue are of the EXACT same quality"?

Look at metacritic, anything that scores below 70 or so is considered to be a flop. And this indsutry lives and dies by metacritic. People's bonus for a game they worked on for three years depends hinges on a metascore that meets the demands of a CEO and shareholders.



Conclusion


This situation sucks. It was blown out of proportion. The guy played through the game on casual mode, and his review reflected that. When called on it, he removed the blurb that proved the developer's claim. Although the reviewer came forward to explain himself, he'll never again be fully trusted - not just by readers but by publishers and developers. And for what reason? He probably played through the game on casual to meet a deadline, so that IGN's review could be first, on a game that was already released, which forced him to take the easy way out.

To say it's not his fault would be wrong, but to say it's all his fault would be just as wrong.

For another inside look at the review process, be sure to check out our 12 Steps to Video Game Reviews.

Comments
  • Masterwabbit
    Masterwabbit

    Joined: Mar 2009
    Posted: Oct 27th, 2011 at 3:06 pm
    I tend to trust the reviews on this site not just because they're thorough, but because they can be funny to read. It's obvious that the people who write here love games and aren't afraid to poke fun of a game's flaws. It also comes in handy for blasting apart games that are downright awful (the Survivor review is still my go to place for a cheer-me-up). I also like how very few games go past a B+ without contributing something new to the industry. Granted, I don't agree with every single review I've read on here, but at least the writer makes his case for what gets a "C+" instead of the "B-".
  • Guernica
    Guernica

    Joined: Mar 2009
    Posted: Oct 27th, 2011 at 3:48 pm
    That does seem to be a sign of quality for the reviews here. Especially when a review is given a fun format. The dx:hr review comes to mind. The reviewer not only loves games here but loves his job to some extent to write such a creative view and I think that gives the review more value all around... even to the quality of the score given whether it be good or bad.
  • Bras
    Bras

    Joined: Jul 2008
    Posted: Oct 27th, 2011 at 4:05 pm
    We deserve more Survivor: The Interactive Review type reviews!

    I must admit that I don't buy a game that's rated lower than B on GR, unless word of mouth is great about it, or I'm just a fanboy of the series/devs/whatever.
  • LawnGnome
    LawnGnome

    Joined: Apr 2007
    Posted: Oct 27th, 2011 at 3:38 pm
    You should just create a new site called Game Establishment, have all your review copies sent to it, and give everything an 8/10. Then you can post the real reviews & scores on GR on time.
  • Guernica
    Guernica

    Joined: Mar 2009
    Posted: Oct 27th, 2011 at 3:45 pm
    Game reviews aren't the only place the 0-100 scale is like that you know. Think of school grades. If I have a 70, then I'm average. Anything above that is above average anything below is ****... You can't pass school with less than a C. It's mostly because of the view of people I think. There is that expected line that we should cross (70) if we don't then it's just not good enough and that's because well... someone else did get above 70. I am not completely sure this is something bad though. If there isn't a line how do we gauge ourselves. We can make the line higher or lower but then that just skews how the "reviewer" gives the grades. But that's just my incoherent babbling 2cents.
  • Rinnon
    Rinnon

    Joined: Nov 2005
    Posted: Oct 27th, 2011 at 4:17 pm
    I think you're right about WHY people view it this way, but I think it's a problem none the less. School marks and Game marks aren't actually measuring the same thing. When we review a game, we are assessing the quality of the game. Marks in school are an indicator SOMETIMES in quality of, say writing, but often in quantity of things remembered during the class. To say you've truly learned everything in the class, you should be over 90%. If you're not there, there are things you seemingly didn't learn. 50% means you only learned HALF the material in the class. We're right to say, that's pretty bad.

    But when it comes to assigning a level of quality to a game... it seems like the bottom half of the out of 10 is just useless. There's no difference between 1 and 5 essentially, except HOW bad a game is. If you were to use it to be more clear, 1 would be "The Worst Game Ever" 10 would be "The Best Game Ever" and 5 would be "Neither Good, nor Bad." As it stands now, where really only 6 to 10 is used, you might as well just be marking out of 5 and be giving a lot of 2.5s. That would at least be more honest looking.
  • C_nate
    C_nate

    Joined: Apr 2009
    Posted: Oct 27th, 2011 at 4:32 pm
    ^ This. Well said.
  • De-Ting
    De-Ting

    Joined: Nov 2006
    Posted: Oct 27th, 2011 at 3:49 pm
    In a way it makes it easier for us. If you don't see a review for a game on GR, then you can probably deduce that the publishers either knew it was rubbish, or they just don't have connections, GR just hasn't bothered to purchase it. However, if the publisher has faith in their game, they'll send it to GR.

    If you think about it, GR has become a prestigious site. It takes a lot to make it big, here.
  • Doc_Holliday
    Doc_Holliday

    Joined: Nov 2005
    Posted: Oct 27th, 2011 at 3:57 pm
    The problem is gaming is becoming a big deal, so media surrounding it is also increasing. Now we have a profitable business in gaming journalism.

    We have a bunch of over-educated, non-gaming people getting into gaming journalism who think that because they played Mario on NES they are true "gamers".

    This is why I've always come to GR; you are gamers, who play and review games because you love and appreciate the art of gaming.
  • Rinnon
    Rinnon

    Joined: Nov 2005
    Posted: Oct 27th, 2011 at 4:08 pm
    So Anthony, you were talking about how some games arrive early, some games arrive late, and some not at all. I was just thinking, it would be neat to make public that information on a regular basis. For example, each week list every game that is released, and which ones you already have review copies for, are expecting review copies for, and which ones you know just aren't coming. Not that we should infer any of those games are good or bad based on this, but it would be an interesting manifesto to read each week. Some readers with a little more ingenuity than myself might be inclined to write up a blog correlating the average scores of games next to their timeliness on arrival. It would be fun.
  • NickKmet
    NickKmet

    Joined: Jan 2007
    Posted: Oct 27th, 2011 at 10:52 pm
    this. yes. I think it would be interesting to see this information. also, i guess that would let us know what games you expect to be reviewing too, which would be nice.
  • NickKmet
    NickKmet

    Joined: Jan 2007
    Posted: Oct 27th, 2011 at 11:06 pm
    this. yes. I think it would be interesting to see this information. also, i guess that would let us know what games you expect to be reviewing too, which would be nice.
  • TurinAlexander
    TurinAlexander

    Joined: Sep 2006
    Posted: Oct 27th, 2011 at 4:10 pm
    Honestly, I don't care one whit about this guy's review. I don't visit IGN because quite frankly, I don't feel that their reviews are particularly honest. I come to GR because, while I may not always agree with your reviews, I know I can trust you guys to give an honest and straightforward review.
  • C_nate
    C_nate

    Joined: Apr 2009
    Posted: Oct 27th, 2011 at 4:29 pm
    This is some of the things I mentioned in the original article and your insight and candor about the process is refreshing. It's that kind of honesty that has been bringing me back to this site for many years.

    My question is, do you think the way it works now will last or will it even get worse? It seems like publishers are getting a lot more strict with who gets to review their games early and how they do it.
  • Longo_2_guns
    Longo_2_guns

    Joined: Jun 2003
    Posted: Oct 27th, 2011 at 5:08 pm
    The other problem with video game journalism is that while there is a lot of competition, there's far LESS mom and pop style competition than, say, movies. While every newspaper reviews movies, there are only a few video game reviewers. So it's far easier to bribe out the big name reviewers than with movies and the like.
  • tinymhg
    tinymhg

    Joined: Jun 2011
    Posted: Oct 27th, 2011 at 5:54 pm
    Cynical as always Longo. BTW how do you think Anthony paid for his house?
  • Commiebot
    Commiebot

    Joined: Mar 2007
    Posted: Oct 27th, 2011 at 5:58 pm
    I just assumed sexual favors and illegal animal smuggling. How else am I expected to get a ferret in California?
  • 213EDD
    213EDD

    Joined: Sep 2007
    Posted: Oct 27th, 2011 at 6:37 pm
    Stick to gerbils much safer.
  • TheJx4
    TheJx4

    Joined: Jun 2011
    Posted: Oct 27th, 2011 at 5:16 pm
    Reviews should never have been averaged in the first place.
    People should just play games.

    We'd be able to do that if the prices weren't so damn high. There are so many games I've been on the edge about, that I would have purchased if it weren't for a $60 price tag. Seeing a few bad reviews doesn't help. If it were $40, I wouldn't even look at a review.
  • Heath_Hindman
    Heath_Hindman

    Joined: May 2011
    Posted: Oct 27th, 2011 at 5:29 pm
    Something's wrong with our eeeeeyes.
  • Anthony_Severino
    Anthony_Severino

    Joined: Oct 2010
    Posted: Oct 27th, 2011 at 6:17 pm
    I'm glad you caught the reference ;)
  • 213EDD
    213EDD

    Joined: Sep 2007
    Posted: Oct 27th, 2011 at 6:29 pm
    That took me actually using my Safari Reader but i ****ing pulled it off just for you Anthony_ Severino. TL;D(Did)R
  • Imnickson
    Imnickson

    Joined: Jul 2006
    Posted: Oct 27th, 2011 at 6:45 pm
    I ran into this issue the other day. I love Dark Souls so I was just browsing for reviews just to hear praise about the game and to laugh at people that thought it was too difficult. Then I came across the review below and my mind was blown. Not only did the guy not complete the game (he only beat the second boss) but he states complete fallacies about the game. I hate posting the site because out will get more hits but things like this need to be learned and never repeated by anyone.

    douglasville.patch.com/articles/dark-souls-yo​u-died
  • hopiamani
    hopiamani

    Joined: Oct 2011
    Posted: Oct 27th, 2011 at 10:57 pm
    I've been going to this site since 1997. I was enticed by the "cheats" page, but I've stayed because of the reviews.

    P.S. I also love the mailbags by Duke
  • lifewish
    lifewish

    Joined: Jun 2011
    Posted: Oct 28th, 2011 at 7:52 am
    the whole review system is a joke but at the same time, it's not going to change.
  • friggest
    friggest

    Joined: Mar 2008
    Posted: Oct 28th, 2011 at 9:54 am
    Game Revolution gave Fallout 3 a "A" which we all know the highest score never given is the "A+"

    So an A+ would be 100% so than the A is actually 95% so your scores are 14.3 points lower than everyone else.

    I love GR's reviews; they are unbiased and critique the game quite well. Ive been a faithfully GR fan since I was 11 or 12. (used them for cheat codes to be honest, but now I never cheat...strange). Im now 25. Keep up the good work.

    Sidenote: I want this read last so ppl would focus on the first part, but I hated Fallout 3. The game was buggy, slow, and drab. The load times killed it and the open world wasnt really open as you were stuck in subways and fallen buildings. But I loved, absolutley loved Oblivion. I stole everything in the game ans sold it (that could be sold) before I even advanced the story.
  • Eddy_DS_Fettig
    Eddy_DS_Fettig

    Joined: Jul 2011
    Posted: Nov 2nd, 2011 at 12:07 pm
    I've been harping on many of these topics for a while now, and I'm glad to see someone else addressing it, too.

    If I were to summarize what the game review industry needs, it's transparency and honesty. Let the readers know the conditions under which the game was reviewed. My second main complaint would be a rehaul of the rating system, but good luck with THAT one. Deciding to use the same gradign scale as the school system was a retarded move on whoever's part that was to begin with. The five-star system is what is more commonly used in critiques in most every ot6her medium, and there's a reason for that: the whole scale actually seems to get used that way.

    Sadly, X-Play on G4 uses this scale when practically no one else does. X PLAY ON G4, people. If THEY are doing something right, what in the hell is wrong with the rest of us?

Post a Comment
LOGIN or REGISTER to post a comment or rate this article.



More On GameRevolution