Steam - Review System Changes

[Silver]

Steam have updated the review system to better reflect how games are as a current experience rather than have old reviews front and center.

-Valve
One common theme we've been seeing in customer feedback about the Steam review system is that it isn't always easy to tell what the current experience is like in a game months after release. This new set of changes released today is designed to better describe the current customer experience in those games. We do this by better exposing the newly posted reviews and by calculating a summary of those recent reviews.

Visibility For Recently Posted Reviews

While there are plenty of new reviews posted every day, we saw that it was often difficult for newer reviews to be seen and voted on enough to become listed as most helpful. As a result, the most helpful reviews presented on a store page would often describe an outdated view of a game that might have changed dramatically over the course of Early Access or post-release development. By listing recently posted reviews more prominently and by defaulting to recent helpful reviews, Steam can now show a more current idea of what it's like to play the game now.

Recent Review Score

Another problem we identified was that review score that appears at the top of a product page didn't always reflect the dynamic nature of the game. For that review score, we'd previously only been compiling an overall score using a simple calculation of the percentage of all reviews that were positive. This let us be really transparent in how the score was being calculated, but didn't accommodate cases when a game has changed a lot (for better or worse) over time.

To address that, we've now added a Recent review score that calculates the positive percentage of reviews within the past 30 days (as long as there are enough reviews posted within those 30 days and as long as the game has been available on Steam for at least 45 days). The overall score is still present as well in case you still find that information helpful.

...

More information.

 

[Lynchden]

Better yet would be to not let people with five minutes of play time post reviews.

 

[Drithius]

Not sure I like this - it further caters to an increasing theme of "Early Access" alpha/beta releases, diminishing the need to put out a competent product on day 1 even more.

 

[suibhne]

Not sure I like this - it further caters to an increasing theme of "Early Access" alpha/beta releases, diminishing the need to put out a competent product on day 1 even more.

I guess you mean that products will no longer be "punished" for a dismal Day 1 release, if the devs get their act together and, after 10 patches, turn the game into something extraordinary - is that it? But I'd suggest that the latter information is vastly more relevant to me as a buyer now, today. The former info is helpful for evaluating the devs' track record for delivering a worthy release-day product, but it has zero to do with whether the product eventually reached a state worthy of purchase and play.

I think it's generally good to provide visibility into both slices of data, but the game's current state is really the only thing I care about if I'm evaluating a purchase now.

 

[Drithius]

Except the state of the game 6 months post-release should not differ so greatly from the state of the game at launch as to make null and void old reviews. If it does, your personal contentment only exists because the publisher/developer exploited thousands of consumers and had them beta test an inferior product at launch. So, yes, they should be punished for such policies rather than rewarded with a forgetful review mechanism.

 

[Terry]

Except the state of the game 6 months post-release should not differ so greatly from the state of the game at launch as to make null and void old reviews. If it does, your personal contentment only exists because the publisher/developer exploited thousands of consumers and had them beta test an inferior product at launch. So, yes, they should be punished for such policies rather than rewarded with a forgetful review mechanism.

Pfff what reality do you live in? If you are only talking about AAA multi-million dollar games you may have a point but for thousands of indi releases that are well supported and fine tuned for months after release then this is an excellent idea.
The little guys don't always get it right at first but at least they keep trying.

 

[KaosWarMonk]

I regularly switch to recent when looking at reviews. What a game was like at release or during early access isn't particularly relevant to me, I want to know what it's like now I'm considering it. The downside to looking at recent versus most popular reviews is you sometimes need to scroll through a few pages to get something more than a line or two.

Having some sort of ongoing punishment by putting up old reviews of a botched release in the first instance is pointless. I've seen many games have a less than stellar release and then go on to great things. And often if those games hadn't been released when they were they would never have seen the light of day.


-kwm

 

[Drithius]

You're missing the point. If a game requires beta testing, that's well and good. But don't put it up on Steam, charge full price, and then expect a market to casually forget that you charged them to beta test your product. That's what Steam's become lately - a goddamn beta testing platform. And people give it a pass... why? Because it's become standard practice. Well, I don't give a crap about standard practice when such practices are idiotic and exploitative.

 

[Dean]

If people would stop buying alpha/beta code released games, publishers would stop putting them out that way. People with patients don't have to worry about such things.


Heck, now days it's part of the QA process.

 

[Dean]

You're missing the point. If a game requires beta testing, that's well and good. But don't put it up on Steam, charge full price, and then expect a market to casually forget that you charged them to beta test your product. That's what Steam's become lately - a goddamn beta testing platform. And people give it a pass… why? Because it's become standard practice. Well, I don't give a crap about standard practice when such practices are idiotic and exploitative.

It's not exploitative if you are an educated consumer. Anyone that buys an early access game on steam should realize what they are getting themselves in to. It's not the publishers/developers fault if people have unrealistic expectations of a knowingly unfinished product.

 

[KaosWarMonk]

Wa? Your grievance doesn't make any sense to me. Whatever.

I like the idea of seeing recent reviews in the first instance.


-kwm

 

[Kordanor]

Checked the 3 Games where I expected the biggest impact:

Skyshine's Bedlam:
Previously: Mixed / 66% positive
Now: Very Positive / 85% Positive

Blade of Destiny:
Previously: Mixed / 50% positive
Now: Mostly Negative / 30% positive
(didn't expect that one)

Jagged Alliance Flashback:
Now: Mixed / 50% positive
(for some reason it doesn't have two scores)

 

[Silver]

I've been looking at some of the kickstarter rpgs to see how they fare with this system

Wasteland 2
Recent: Mostly Positive 70%
Overall: Very Positive 84%

Divinity Original Sin
Recent: Very Positive 90%
Overall: Very Positive 88%

Pillars of Eternity
Recent: Very Positive 88%
Overall: Very Positive 88%

Dead State
Recent: Mostly Positive 70%
Overall: Mostly Positive 73%

Shadowrun Dragonfall
Recent: Very Positive 85%
Overall: Very Positive 92%

 

[Silver]

Checked the 3 Games where I expected the biggest impact:

Jagged Alliance Flashback:
Now: Mixed / 50% positive
(for some reason it doesn't have two scores)

You only get a recent score if there have been enough recent reviews during a period of time. In other words no one is buying it.

 

[Dean]

Agreed, I like the changes Steam are making to the system. I want to know what state the game is in when I'm ready to purchase it. Don't care what shape it was in 4 months ago. The changes make it easier for me to decide if I want to make a purchase, or if I should wait, or just forget it and move on.

 

[you]

I think the flaw is folks most likely to enjoy buy early and more causal later hence a quality game likly to drop under new system. I suspect RPG will be most vulnerable. This presume the period after major patches/enhancement. I expect games like dragonfall to be most vulnerable. Am a bit surprise by wasteland large drop.

 

[lostforever]

All in all, this may be a good decision from my point of view since I hardly buy many games on release these days. If I am buying a game on release day, then that decision was made outside of steam reviews. So really I want to know the sate of the game when I am buying it and not when it was on early access etc.

I don't really agree that the day one release quality of games these days are bad compared to the past. In fact I think its the opposite. I haven't had any troubles with new games in the last 5 years. I remember fiddling with device drives and the like to get games working in the past. There used to be constant crashes and the like. Things seems to work fine these days for (touch wood!) me. I know my personal experience isn't reflective of others but I am happy with the state of games these day compared to before!

 

[Archangel]

I think this is a good change. A RTS that I like already got rewarded for its fixes in last 6 months by its score jumping from 66% to 77%.

 

[joxer]

Both old and new system are flawed.

1. There is no player merit, a review coming from a person that owns just one game on Steam is valued the same as a review from a person who bought over 100 titles, that leads to easy abuse/boost the score from publisher

2. Concentrating on month old reviews is rediculous. If a game didn't get any OFFICIAL patch during half a year, what's the point of dismissing 2-3 months old reviews? Oh wait, the publisher can put (quite) a few reviews monthly so again we have #1 case

3. thumbup review from a person who played a game for less than hour? kill me but IMO it's a fraud

4. thumbdown review from a person who played a game for less than hour should be at the top of everything - these are usually caused by bugs, crashes or inability to launch a game

5. Steam doesn't remove a review when the game got returned and person got refund, but IMO it should

6. A review that's basically a spam sentence and nothing more said? These should be disabled instantly.

 

[fadedc]

I like this change, especially since you can still look at the overall review if you think that's more helpful. I think there are already significant negative consequences for a bad release. The initial release of a game is when they make the vast majority of their money. If that release is bad, it will really hurt the company. But when I'm trying to decide if I should buy a game, I'm not interested in some concept of justice or retribution for a bad release, I only care about whether the game is good now.