Metacritic scores on Amazon can hurt gamers, publishers, and writers

If you’ve been on Amazon lately to order a new game, you might have seen a Metascore number to the right of it. In an interesting move, online retailer Amazon has implemented scores from Metacritic next to the games on their site to give consumers an idea of how well a game is rated by critics and users.

Appearing to the right of the game, this graphic features the official Metacritic rating, aggregated from dozens of press reviews averaged on a 100-point scale. And when you hover the cursor over the box, it displays the user rating from the site as well. The only other online retailer to use Metacritic is Steam. The average gamer and consumer might think this is a cool feature–a simple glimpse of how good a game is based on professional reviews. For a publisher or developer, though, there are serious downsides.

If a game is not necessarily rated highly at launch due to some game breaking bugs, but is later fixed by patches and achieves its maximum potential fun and stability, it will still have the rating from when it launched, pointing customers away with a score that doesn’t reflect the game’s current state. Fallout: New Vegas is a prime, infamous example. Shortly after its 2010 release, New Vegas’ myrid bugs could cause your game to freeze at any moment, causing lost progress and ruining the gaming experience of many. Edge Magazine, who gave the lowest score (60 out of 100) for New Vegas on Metacritic, says, "Creatively, New Vegas gets almost everything right. Mechanically and technically, it’s a tragedy. So, it’s a simultaneously rewarding and frustrating game, the gulf between what it is and what it could be a sizeable stretch indeed.”

PSU was a bit more generous, citing "crippling glitches and bugs" and "long load times" as points against otherwise excellent gameplay, world design, and emergent storytelling. PSU gave New Vegas an 8.0 out of 10–80 out of 100, in Metacritic’s scale.

Both reviews could have turned out very differently if Obsidian had done more beta testing and fixed bugs so there was no need for patching. And today, New Vegas is a very different game–stable, almost entirely bug-free, and a hell of a lot of fun. Today’s New Vegas bears little resemblance to the New Vegas of 2010. A Metascore of 60 utterly fails to describe the current experience to first-time buyers.

Besides Metacritic’s failure to keep up with a game’s ever-evolving state, the mere combination of scores from multiple review sites creates problems in its lack of exhaustiveness. Metacritic uses a weighted average: "we assign more importance, or weight, to some critics and publications than others, based on their quality and overall stature.” Suddenly, Metacritic is no longer an impartial judge. A score from a lesser-known website is valued less toward the weighted average than PSU, Destructoid, IGN, and other contemporaries. This cloudy judgment fails to account for the fact that there are brilliant writers and brilliant games criticism–often more genuine and discerning than mainstream outlets–in hard-to-find places with smaller audiences. The average star rating of a game on Amazon tells you more than the Metascore; at least Amazon isn’t disregarding some reviews based on an algorithm.

Metacritic is a tool, not the end-all, be-all games criticism generator. There’s some validity in gamers quickly checking the Metascore when deciding to pick up a new release, but in my opinion, the detailed opinions of professional and amateur writers–and a working knowledge of gaming’s current trends and patch histories–is far more valuable. Staying abreast of video game news tells you what’s good, what’s important, and what’s not technically "good" but deserves to be played anyway–even as a game’s quality changes over time. Hopefully, we are not heading in a direction where people skip past a thoughtful review of a game, or recent news articles with relevant, timely information, just to make a hasty judgment off a convenient, but flawed, number.

What do you think of Amazon’s decision to post Metacritic scores next to games? What’s Metacritic doing right, or wrong? Give us your take in the comments below.