Gamasutra - The Effect of Reviews on Game Development

Gamasutra posts an op-ed from Lewis Pulsipher, former game designer and current teacher of game design, discussing a panel discussion called "Teaching to the Test" from the Triangle Games Conference on the effect of reviews on development of games:

...To explain the title, K12 teachers tend to teach what is on end-of-grade tests to the exclusion of almost everything else.  The panel considered how much game development studios and publishers create games to meet the “test” of reviews...
Their answer to the main question was “definitely not,” though they do pay attention to what individual game fans say on forums and email..Benito saw fan opinion as more "pure from the heart" than the reviews...

On the actual review process:

Panelists clearly did not care for reviews in general, probably because they felt so many were poorly-written and often contained mistakes.  One panelist specifically referred to the reviews on IGN and Gamespot as “white noise”, and all panelists clearly felt that reviews are often “subjective” rather than “objective”.  Of course, “subjective” can be just as accurate (in fact, more accurate) than objective, depending on the situation, the problem is that reviewers don’t explain their biases and why they feel as they do, so readers have no basis to judge the opinions. 

Some reviewers clearly don’t understand how reviews, of any medium, work.  They should answer three questions:

• what were the creator(s) trying to do
• how well did they do it
• was it worth doing

To answer these questions they must explain “why”, not merely say “this is a piece of junk” or “I don’t like the graphics” or “what a dumb idea”.  But this makes reviewing more difficult, more work.
One panelist suggested reviewers ought to “take a step back” and watch others play the game, in order to acquire more than one point of view.  They also need to put themselves in the shoes of a person who’s saved his pocket-money to buy a game, as opposed to reviewers who have piles of freebies to try out.

Reviewers who assign an actual numeric evaluation should provide several scores for different types of players, e.g., hard core, casual, RPG fans, shooter fans, whatever is appropriate to the audience.