MEMBER BLOG

Game Revolution Exposed: Grade Padding Scheme Revealed!
Posted on Monday, May 9 2011 @ 11:35:41 Eastern

Gamerevolution.com has been a fixture in my life for over 10 years. I was around when the only way to enter the site was through a compacted car and the home page could fit on an 800x600 postage stamp.  I even remember when the mailbag was a weekly feature! And when it was a biweekly feature. And a bimonthly feature.  And a quarterly feature. And when it became less a ‘feature’ and more a ‘fat chick you try to usher through your dorm hallway as quickly and quietly as her drunken stumbles allow, but oh god she just can’t plant those lush feet without summoning the God of Thunder, so you stuff her in a broom closet for three years running and pray no one ever needs a dust pan’.
 
I may be bitter.
 
Point is, I’ve been around, and you can take it from me when I say that the critical nature of reviews at Game Revolution has evaporated like morning rain under midday sun. Sometime around Game Revolution’s purchase by Bolt Media, an executive came down from on high and stuffed ethics into the broom closet next to a then frightened and confused mailbag. Now, both lie decomposing while new games are wetly gummed by a novice pack of reviewers too well-sated from publisher kickbacks to wield the fangs of objective criticism that once protected we consumers so long ago. Game Revolution gives out more ‘A’s than Oakland Coliseum on free hat day, whereas the only places capital ‘F’s can be found are in the titles of reviews like Far Cry 2 and Fallout: New Vegas. Both of which, incidentally, received an ‘A-‘.
 
But maybe, dear reader, you don’t want to “take it from me” that your beloved independent game review site has been slit wide open and re-sewn into a corporate hand puppet. Maybe you want cold, hard proof that since the buyout of Game Revolution in late 2004, the average review grade for games has experienced a dramatic and otherwise inexplicable inflation.
 
Proof you want, proof you’ll get.
 
To gather the data for this exposé, I turned to Game Revolution’s own database of reviews. Since its first postings on June 5, 1996 (an inaugural class consisting of PC games Alien Odyssey, Duke Nukem 3D, Mechwarrior 2, Tomb Raider, War Wind, and Warcraft II) to May 1, 2011, Gamerevolution.com has reviewed 4,193 individual games—almost 5.4 games per week. This number excludes 40 Mac game reviews, 13 reviews with no legible grade, and a few double posts on the PC and PlayStation lists.
 
“But wait,” say the Game Revolution sycophants, “what about multi-platform releases? If you count 3 ‘A’s for games like Fallout 3 and AC2: Brotherhood, they will unfairly weight the average review score. You *******.”
 
While I might be an *******, I’m hardly unfair. For each multi-platform game with multiple reviews, I removed duplicate grades. So, for Madden NFL 06, the three ‘B+’s for the Gamecube, Xbox, and PS2 reviews are reduced to a single count, but the ‘B-‘ for the PSP review and the ‘C+’ for the Xbox360 review are both included. Condensing all the multi-platform reviews, we arrive at a total of 3,516 individual grades spanning nearly 15 years.
 
Gamerevolution.com was purchased by Bolt Media, Inc. on November 17, 2004, and then purchased from a bankrupt Bolt Media by CraveOnline on February 25, 2008. I will consider the ‘self-ownership’ base period to be any time before the Bolt Media purchase. With the final data table tabulated, all that’s left to do is convert the grades into numerical scores, compare the averages across each time period, and find out if the periods under either owner yielded (statistically) significantly higher scores than the self-ownership base period.
 
“But WAIT, you *******!” chirp the sycophants. “You can’t just convert grades to numbers arbitrarily! If Game Revolution gave out relatively more ‘B+’s during the self-ownership period but relatively more ‘A-‘s during the Bolt Media period, it will make a HUGE difference if you score a ‘B+’ as an 89 and an ‘A-‘ as a 90 versus scoring a ‘B+’ as an 87 and an ‘A-‘ as a 92. And what if you converted on a strict 0-100 scale like Metacritic used to? You’d botch it so hard that people would mistake you for Umberto Granaglia!”
 
Since I would sooner die than give the shills at Game Revolution even a modicum of wiggle room through which to slither, I’ll placate the sycophants on this one. Instead of converting to an arbitrary point score, we’ll take the percentage of each grade’s frequency in a period relative to the total. So the 173 ‘C+’s given during Game Revolution’s self-ownership period out of the 1,981 non-duplicate grades for that period yields 8.73% ‘C+’s during self-ownership, and so on. Then, we will arrange the grades by relative value (A > A- > B+ > B > … > F-) and create a cumulative distribution function.
 
“A cumulus distributed what?” the sycophants query. For our purposes, a cumulative distribution function shows, for each possible review grade, the percentage of reviews that have a score greater than or equal to that grade. To calculate this value for any given grade, we add up the percentage of reviews for that grade and each grade above it. As we move down the grades from ‘A’ to ‘F-‘, the percentage increases, finally reaching 100% of all reviews being greater than or equal to an ‘F-‘. This is a nice tool to use when we know that an ‘A-‘ is worth more than a ‘B+’, but we don’t want to argue about the difference in value between the two grades. 



Above is a sample data set and cumulative distribution function for an imaginary period in GR history. The green arrows show how to read the graph: for this period, a ‘C’ grade or higher was given to 60% of reviews. We can see from the data set that the difference between two adjacent points on the cumulative distribution function is exactly equal to the percentage of reviews receiving the lower grade. This means that we can judge from the graph what percentage of reviews had a certain grade by looking at the slope between that grade and the previous (higher) one.
 
Got all that? Great. Now forget everything and just remember this: for two different time periods graphed together, if one line is higher than the other across all grades (particularly the ‘A’ to ‘B-‘ period), that line represents the period with more lenient grading.




First up: Self-ownership Game Revolution vs. Bolt Media. We can see that one line is clearly above the other across all grades. And that period of lavish grading excess is…
 
The self-ownership period. Son of a *****. According to Game Revolution’s own review data, grading actually became more stringent during the Bolt Media ownership period. But then again, those Bolt Media guys went bankrupt in 2007. They wouldn’t know a game review website from a giant pile of anti-money, and those lower review scores could easily have come from equally low reviewer morale under such incompetent managerial overlords.
 
My issue lies with the current Game Revolution review team and their owners, CraveOnline. And if we compare the three years and change under Crave’s ownership to the independent period, we’ll find—



Holy ****.
 
Yes, those are two individual cumulative distribution functions. Both periods’ grade distributions are just so closely matching that the lines actually overlap each other. If I had to save a man from hanging by proving that the percentage of reviewed games receiving a ‘B’ or better from Game Revolution was constant over the periods of self-ownership and CraveOnline ownership, this is the chart that I would present to the jury.
 
So whether self- or corporate owned, with an old crop of veteran reviewers or freshly conscripted foot soldiers, Game Revolution has been scrutinizing games with the same honest, unflinching criticism we’ve all come to know and love. The good are praised, the bad are razed, and all advertising kickbacks be damned straight to Hell.
 
Don’t I feel like an *******.
 
BONUS: For those who want to see a more detailed comparison view of the actual percentage breakdowns for each grade, have one and the other.