The Review Score Debate

Ever since the Uncharted 3 review score debacle that ridiculously took place last week, I’ve been struggling within in regards to how I want to proceed with review scores for this site. I personally hate review scores. I think too many people look too far into what they possibly mean, especially when the scale includes a decimal system beyond .5. I also think that these same people rarely actually read reviews and simply glance at the score and use that as their fuel for comment fodder.

Back when I was writing for the now defunct Kombo–may it rest in peace–I was a “victim” of readers not reading my review and basing their hatred of me from a score. The most vivid memory was when I gave Final Fantasy XIII a 9 out of 10, and I wrote nearly 2000 words explaining exactly why I gave it such a score. From the first few comments, it was readily apparent that no one read my review, as they insisted I didn’t consider A or B when I clearly mentioned A and B in the review.

So then there’s this site. I implemented a review score system because I wasn’t sure how else to handle reviews. I was more worried about having an audience in the first place than I was about my personal grumbling with review scores. And I have to admit…I struggle every time I have to place that score on a review. Unless a guide was absolutely worthless or absolutely perfect, I had difficulty coming up with a score. And it wasn’t just the score; I had doubts that my words backed up the score enough. I have in my head how the 1-5 scale works, but sometimes it all gets fuzzy. It did this really well, but it failed here, so that prevents it from being perfect. The design was bad, but it was easy to look something up. Everything was great, but it was wrong about this one thing.

See what I mean?

The natural lean, when I debated this with myself and on Twitter yesterday, was to stop giving numeric scores, be a part of some sort of movement to abolish review scores from gaming, and create some sort of simple word scale. For example, Gaming Angels works on a Buy/Rent/Pass scale, something I really like. So I proposed on Twitter a review scale of Buy/Borrow/Burn (in homage to the Mortal Kombat strategy guide video review), and I was completely ready to run with that…until this morning.

I started to think of everything I consider when I review a strategy guide. To me, it goes beyond the simple recommendation of buy or pass, it’s about how well the book is put together in the first place. The big example that comes to mind is the strategy guide for The 3rd Birthday. That guide was so well done, I’m not sure I could just say “Buy” as its final score.

Then there are the online guides. I would have to create a separate scale for those, something I need to do anyway, but how would a “word-based” scale work? Do I compare them to the print guides? Do I call it worthless or great? Then that sends me down the road of wondering if I should review online guides at all, since they can be changed at any moment in time.

So there’s my inner debate, and I’m not sure there’s a “right” answer. Maybe I’ll just go to letter grades, like what we got in elementary school. But then there’s the potential of adding pluses and minuses…ugh, I’ll never win.