About Us | Game Reviews | Feature Articles | Podcast | Best Work | Forums | Shop | Review Game

Consoleation: The numbers game

Peter Skerritt's picture

Gears of War 3 Screenshot

There has been a significant amount of griping over video game review scores recently. Reviewers seem to be on the defensive to readers, publishers, and developers as they attempt to further justify a number that 1,000+ words apparently could not. Games with review scores less than an 80 average on Metacritic are presumed to be bad (or worse), leading to weaker sales and denied bonuses from publishers to developers. A number—not words, opinions, or viewpoints—has become the sole focal point for a game's quality and the discussion revolving around each game.

It's an unfortunate trend that we've allowed ourselves to fall into, in all facets of the community.

I understand that video game reviews serve different purposes to different people. My notion that reviews serve as a tool to assist with purchasing decisions carries over from the days before the World Wide Web and smartphones, when video game magazines like Electronic Gaming Monthly, Video Games & Computer Entertainment, GamePro, and others were the only way we could get any realistic feedback about an upcoming release. There were obvious purchases, of course, but many other games weren't easy to decide on. By taking a look at a couple of each month's gaming magazines, I was able to devise a budget and decide which games to buy and which games to rent or pass over. If there were the theatrics that we see now—such as blacklisting, firing reviewers due to low scores, review embargoes, and other questionable events—readers like me were oblivious to them because they weren't public.

There were numbers back in those days, of course. Some magazines had shorter reviews (like EGM, for example), and others went with longer pieces. The number was only part of the equation back then. The words meant something. Perhaps this was because there weren't thousands of video game magazines, as compared to the huge number of video game review portals and websites that we see today. We read reviews because that's what we had until the following month.

The landscape is different now. It's all about the number, and the words are secondary. The games themselves are tertiary.

Just in the last couple of days, I've seen content from Jim Sterling, Patrick Garratt, and even gaming comedy act Mega64 about the drama surrounding review scores. While the pieces are all interesting (and even amusing at times), the controversy surrounding review scores is pulling attention away from what are supposed to be the main attraction right now. What about the games? In a span of less than three months that has provided so many quality experiences, we're talking about numbers. It's not completely dominating the discussion, of course, but the fact that the topic is sharing the spotlight at this time of year is disappointing.

Discussions pertaining to how to fix the review process have been ongoing for some time, but we're just spinning our wheels. Dropping review scores is an ideal, but isn't realistic. Reviews without scores attached to them tend to have lower credibility than reviews that do, and publishers actively support review portals with scoring to satisfy various metrics. It's also not feasible at this point to demand a wider scoring scale after months of accepting the 7-10 scale that Metacritic has set forth. Rewriting the review scale would lead to potentially damaging results for publishers, developers, and gaming press alike. We can say all we want that 5.0 or 50 is average, but Metacritic just doesn't see it that way. Arguing that Metacritic has too much power won't change anything either, given the importance that all facets of the industry have allowed the aggregator site to build.

I'm not sure that there's a fix for the numbers game, which is a shame because there are lots of great words being written behind those numbers that are generally being ignored. The words are, at least in theory, what keeps the numbers game from being arbitrary… but if nobody reads them, what's the point of the process? That's arguably the biggest flaw of all, and until we can teach people to slow down and equate those words to the numbers like we did years ago, the controversy will never go away. The numbers will win.

Category Tags
Platform(s): Xbox 360   Wii   PS3   3DS   Nintendo DS   PSP   PC  
Articles: Columns  

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

The main problem with this

The main problem with this industry is that inexperience and plain idiocy can win through; the actual good writers get lost in a sea of mediocrity, which is why this site isn't as big as, say, Gamespot.

In contrast, the film industry is a lot better and more reliable with reviews; Rotten Tomatoes is actually pretty reliable -- especially if you filter down to 'Top Critics' -- and is clearly not influenced by film-makers what-so-ever. Since gamers are typically young and dumb (and they are, as far as journalism goes) the corruption and higher scores for more publisher cash/backing runs rife.

The biggest problem at the moment, aside from the obvious corrupt reviews, is inconsistency; one reviewer will criticise a game for one element in a game, but completely ignore it in another purely because 'they like it more'. It's kinda annoying. I'm glad there's guys like Brad and Mike on here (others too, obviously), and Jim Sterling and Yahztee elsewhere, otherwise reviews would be completely pointless right now. And I mean that.

The video game score

The video game score inflation debate is the same as the grade inflation debate in secondary education. One side argues that scores/grades are inflated and don't mean anything anymore, the other side argues that score/grade inflation is a fact of life and that the only solution is to keep scores/grades inflated, because everybody else is inflating the scores/grades.

I see merit to both arguments.

Scores mean little to me. The reasons for the scores are much more important. For that reason I generally pay much more attention to the written review, particularly from sites and critics I trust.

Game companies have some right to complain about "activist critics" who give low review scores, just because they can. Games like Gears of War 3 have massive budgets and undergo many months, even years of playtesting. They spend this money because they want their game to get good reviews. One low score from an activist critic can throw off a Metacritic score. Why Metacritic scores are so important, I have no idea.

But from a developer's point of view, I can understand the frustration when a game like Gears receives a low score from a reviewer who scores it according to criteria that have nothing to do with the amount of polish and work that went into the game. And I think those developers have a legitimate beef in that kind of scenario.

What's the solution? Give the game a numerical score according to objective criteria, and then put the subjective criticism in the written review? Sounds like a good idea, but I think there's an irreducible amount of subjectivity that goes into any numerical score, no matter what scale you use.

I like the idea of a score average of individual categories (sound, graphics, presentation), but what do you do with a supposedly objective category like "gameplay"?

I dunno :/

There are soooo many review

There are soooo many review sites, we should have a rich depth of opinions and thoughts about video games. But instead, we basically have one opinion that everyone must share (if they want to keep their position as a game reviewer).

Sure, there are smaller titles (where giving them poor reviews will not piss anyone off) those games will get more honest scoring. But for the most part, notable releases get their One review.

The problem with

The problem with scores/reviews aren't the scores/reviews themselves but their integrity. Moreover, a measure other than a number will not change a metascore impression. All gamers understand that a 10 or 9 means a great game, 8 good, 7 average, and anything below lacking. Any other scale is going to ultimately represent the same.

If reviews didn't have an attendant rating they'd be read more frequently, closely and with more interest. That'd be a boon for reviewers who take pride in their craft and probably for gamers as well.

Point of Clarification on Metacritic

Peter, I appreciate how balanced this post is on the topic and thought it covered the issues pretty clearly without pointing fingers and flame baiting.

There is one point I think should be clarified. Metacritic isn't the one setting the 7-10 scale. Their system is a reflection of the industry. If you dig into their FAQs, you'll find this explanation:

Quote:

Virtually all of the publications we use as sources for game reviews (a) assign scores on a 0-100 scale (or equivalent) to their reviews, and (b) are very explicit about what those scores mean. And these publications are almost unanimous in indicating that scores below 50 indicate a negative review, while it usually takes a score in the upper 70s or higher to indicate that the game is unequivocally good. This is markedly different from movies, TV or music, where a score of, say, 3 stars out of 5 (which translates to a 60 out of 100 on our site) can still indicate that a movie is worth seeing or an album is worth buying. Thus, we had to adjust our color-coding for games to account for the different meaning of games scores compared to scores for music, movies and TV.

What's also interesting to note is that this statement changed from when I checked it probably two years ago. At that point, it was explicit in saying that web sites and publications unanimously agreed that 50 or 5 was average. So some where since then, Metacritic decided that the industry standards had shifted.

if you already know that

if you already know that reviews and scoring systems are problematic/unreliable/manipulated, then why continue to agonise about them?

let reviews/scores be what they are - a commercial aspect of the gaming and publishing industries.

there's plenty of honest feedback and reflective pieces on games on the independent sites and blogs that actually write thoughtful critiques. just like you guys here! :)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Code of Conduct

Comments are subject to approval/deletion based on the following criteria:
1) Treat all users with respect.
2) Post with an open-mind.
3) Do not insult and/or harass users.
4) Do not incite flame wars.
5) Do not troll and/or feed the trolls.
6) No excessive whining and/or complaining.

Please report any offensive posts here.

For more video game discussion with the our online community, become a member of our forum.

Our Game Review Philosophy and Ratings Explanations.

About Us | Privacy Policy | Review Game | Contact Us | Twitter | Facebook |  RSS
Copyright 1999–2010 GameCritics.com. All rights reserved.