So it’s safe to say these sites have some work to do, but that isn’t stopping the industry from taking them pretty seriously. At EA’s recent unveiling of the incredible Valve Portal system, Valve overlord Gabe Newell noted the success of Half-Life 2: Episode 1 by using its Metacritic score. That’s unsurprising considering the company's Steam service has Metacritic scores embedded right into its download manager. We’ve seen Gamerankings ratings listed on promotional websites time and again. And then there was our own unfortunate experience with Galactic Civ II.
Luckily, however, most of the industry doesn’t seem too infatuated with the numbers. We talked to several PR people, designers, and brand managers during our research for this piece, and by and large they use these sites the right way: as macro resources rather than accurate converters. Bethesda’s Pete Hines summed up their sentiments when he said, “We definitely use sites like Gamerankings and Metacritic to see how titles are doing overall because they're good resources for everything that's been said about our game.”
We agree, although we'd take issue with the "good" part. Gamerankings provides the bare minimum with its basic functionality and transparency. Metacritic also fulfills the basic functions of a meta site, except that they come to conclusions for you instead of giving you the tools you need to draw your own. And RottenTomatoes…well, we’d like to pretend they don’t matter, but then we Google our own names. Sigh.
And we're happy to remove the chip from our shoulder so long as users, reviewers and industry types understand that current metareview sites are about as scientifically accurate as Back to the Future. The inherent concept is great, but until someone decides to fix the rampant metatorializing, bizarre math and questionable standards, it's truly anyone's game.
Oh, and in case you were wondering, by averaging Rotten, Two Underpants and 50, our aggregate score for all three metareview sites is YELLOW.