The scoop on the GTA IV score
Right, so let's talk about that GTAIV score. For those who haven't seen or heard, here's the deal: there was a bug in our publishing system on Monday that caused the incorrect score for GTAIV, a 9.5, to appear for on some pages for a while. This happened hours before we posted the actual score with the review, a 10. This has all led to a lot of talk and speculation so I'm going to set things straight, short answer: it's not true. Longer answer requires me to run you through stuff which I'll be doing now.
Our review process is like so: Once our reviewer has finished the game, he writes his review, submits it to copyedit, and then goes back over the text to look over the changes copyedit made before submitting it to production who then produce the review. Once the review is produced as a Web page on our QA servers, a link goes out to the reviews team for the peer review process. During that time the review score is not final. I'll say that again for dramatic emphasis: During that time the review score is not final.
In the case of GTAIV, that non-final review had a score of 9.5 attached to it--a score which the peer review process is designed to fine-tune and then set in stone. As has been standard GameSpot operating procedure for years, the last step of the peer review process is a discussion about review content and score. The content discussion entails making sure that relevant gameplay features and whatnot are mentioned to ensure the review is accurate as possible--and that said score matches said review.
As Justin noted in his blog, there were enough people in the reviews group who felt GTAIV deserved a 10 that we all holed up in a room to settle the matter....as the review was in the QA process on our staging servers with a 9.5 score attached to it. Ironically, one of the staff poked their head in the room during the at-times heated discussion and mentioned that it looked like a bug on the staging site had caused the QA score to go live for a split second. The importance of this event didn't sink in at first, since were so embroiled in our talks. Why were we so focused? Because we take matters of review score seriously. I'll reiterate what Justin said in his blog: We don't hand out 10s lightly--we've given out just four in GameSpot's entire history.
I've hit up our tech folks to find out what the deal was. After some back and forth, they got me answers because I wanted to post about this and include some info. So here's the tech nitty gritty for those that care:
* a 9.5 score, which as I noted above was not final, was the original score placed on the GTA4 page and became the page that was cached while testing and reviewing the site for quality assurance.
* Due to a bug in our publishing system that has since been corrected, a cached page containing the score and review blurb, but not the review itself, appeared on the Production version of the site.
* The length of time this was up is not precisely known but logs show that changes were made from 4:45PM PST - 8:20PM PST, with the actual review going live at 8:20PM PST.
And there you have it. A publishing snafu + some tremendously awful timing = needless drama. I'm well aware there's a bunch of folks out there happy to add this to the conspiracy theory tapestry that has been woven about GameSpot. Adding to said conspiracy was a blog post from a moderator saying the 9.5 score was deliberately put up as a "red herring" to test reader reaction. The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1).
I'm disappointed that, end of the day, one of the best-written and best produced text and video reviews is being cynically written off by some. None of the naysayers know the effort that went into making sure our review of probably the biggest--and so far, the best--game of the year was as bulletproof as possible. The personal attacks on Justin are unnecessary. Apparently people don't realize that Justin's been at GameSpot for 9 years--starting with the original GameSpot UK--and has reviewed all sorts of games in that time.
But so it goes. Bugs and assorted glitches are a fact of life on the web just as misspellings and assorted printing issues can be with print magazines. We take our reviews seriously here and we don't post them until they're ready. This is why GTAIV went up when we wanted it to with the score we wanted it to have. For those that gave us the benefit of the doubt and that didn't jump to conclusions, you have my thanks. To the others? ...
Print
Comment
Tag
Facebook
Digg this
Comments
And now we know.. the rest of the story. Thanks Ricardo.
So basically, QA = rough draft.
but, i do have 1 question. How does tha game go from a 9.5 to a 10. Who can decide this. Is this decided by people who have read the review (but may not have necceseraly played tha game yet) and feel that the text reads more like a 10 or do the people who are involved in this decision have to have played the game. How does that work?
In all seriousness, the only problem should be that the score changed, not that it got a 10 instead of 9.5, but you've explained why it was changed, and why the 9.5 was not official, pretty well. The only thing I would add is how often review scores change before going live. I can only assume it happens semi-regularly. If this tweaking of the score happens very rarely, however, I think most people would want to know why. I got the game at the midnight launch (GTA IV, not Big Rigs) and am sure I will love it!
I don't doubt that this is exactly what happened... but you have to admit that GS hasn't had the most "scandal-free" reputation lately. So any little thing that goes wrong, or any major thing that GS does (like hand out a 10) is going to get drug through the mud because there are lots of people out there that just don't trust you folks anymore.
If I were you, and clearly I'm not, I'd find the CNET execs that botched the Gerstmann-gate thing, show them things like this ridiculous little drama, and make them understand how badly that donkey show is still hurting the site.
Yet today it was the voice of many telling one man that he was too critical of flaws, that it was he who over-evaluated their impact on the gameplay and fun. A true indicator of what business is today in group-led problem solving. Many people, one research, one solution.
Perhaps people should learn why a 10 is a ten and learn why either way still produced the same result. A game that cannot be reasonably improved at the time it came out.
Ultimately, it doesn't much matter, and this little ordeal simply highlights the pettiness of people on the internet who invest themselves entirely too much in spewing negativity. I'm not asking for everyone to sing kumbaya here, but damn people, give Gamespot a break would ya?
needless drama indeed. Im sure the infoil hat croud loved this episode
"The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1)."
Heh, ever heard of 1984?