Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
October 25, 2014
arrowPress Releases
October 25, 2014
PR Newswire
View All
View All     Submit Event





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
The Lack of Quality Control and the Race to the Bottom
by Tomasz Mazurek on 08/05/14 07:35:00 am   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

With the recent attempts by Steam to include in their catalog every game that has ever been published we have heard many voices calling for an introduction of some sort of quality control. Those voices are mostly motivated by the care for the customers, who get cheated out of their money by developers selling barely functional junk like Air Control. Of course one can and many have made the counterargument that easier access to Steam is overall good and those players have only themselves to blame, caveat emptor and so on. Now as developer I oppose this view, for two reasons. The first is that for a seller of games, or anything really, "buyer beware" is one of the stupidest things to say, even if said in Latin. The second reason is that I believe lack of quality control can, under current circumstances in the PC market, hurt the indie developers as much if not more than it hurts the customers. In this article I will explain how exactly this might happen.

The Market for Lemons

Have you ever heard about the market for lemons? "The Market for Lemons: Quality Uncertainty and the Market Mechanism" is a 1970 paper by the economist George Akerloff describing a market failure mechanism that he observed in the used cars market. What do used cars have to do with computer games? I think the market failure mechanism described in the paper, known as "information asymmetry" may happen in the games market. In fact, I think that it has already happened in some of the markets and we might be at the verge of it happening in the PC indie games market.

But first, let's go back to the lemons. "Lemon" is supposedly an American slang term for a used car that turns out to be defective only some time after it has been bought. Lemons are an unavoidable problem in the used car market. It is nearly impossible for the buyer to be completely sure he is not buying a lemon - checking for some types of damage, like the wear of internal engine parts, is way too difficult to be practical. What makes matters worse is that an individual seller has no incentives to disclose such problems, since that would lower his car's price. On the other hand, if the seller is sure his car is actually good, there is no easy way to signal this fact to the buyer, since everyone can and does claim his car is in top condition.

Under such conditions the most rational thing the buyer can do is make an assumption about the car's quality based on the average quality of cars in the market. The consequence of this rational behavior is creation of a positive feedback loop between the quality of the cars and their prices. Assume that there has been a surge in the number of low quality cars on the market. Maybe the price of gas has soared and the poorest car owners, who could not afford proper maintenance, now cannot afford driving cars at all, so they are selling them. The result is that the average used car quality falls. This means that buyers will assume the used cars are of lower quality and they will be willing to pay less for them. This can cause the sellers of the good cars to decide that they do not want to sell their cars for such prices - maybe they will drive them for a few more years, or sell abroad. This further depresses the average quality of used cars and so on and so forth. Soon it may turn out that although there are people who would like to buy a premium used car and although there are people who would like to sell such cars, the market only trades "lemons", because everybody assumes that most used cars are lemons. Note, that this state of the market might persist even though the original reason for the fall of quality has long passed. Also note, that this positive feedback loop may start at any stage - at the fall of the average quality, at the fall of prices or at the withdrawal of quality cars from the market.

This "information asymmetry" problem is, obviously, not limited to the used cars market. In fact George Akerloff and Joseph Stiglitz argue that information asymmetry is a norm rather than the exception and they got the Nobel prize in economics for their work, so I think there is something to it. Probably the most known (and the most controversial) other example of market for lemons is the market in private health insurance. I won't discuss that particular topic, due to its rather inflammatory nature.

The Information Asymmetry in Game Markets

Now let's consider whether under certain conditions it is possible for game markets to become markets for lemons. It is obvious that game developers usually know more about their games than the players. It is also obvious that the individual developers do not have much incentives to announce that their game is crap. But games are not like used cars since a game can be, however subjectively, reviewed. Reviews, even if not perfect, at the very least protect the players from buying utter crap. Or do they?

If the market is flooded with games, e. g. due to an explosion of shovelware, many games are not reviewed at all and many are reviewed only by minor review sites that players might not consider reliable. Nothing strange about that - reviewing takes time and nobody wants to review the 255th Flappy Bird clone. User reviews could help, but they are usually far too easy to manipulate to be useful. In fact we have a proof that people are buying games without reading any reviews - because we know there are people who have bought the Air Control and similar games. And the only way they could have done that is if they have not read a single review. So, if people are not reading the reviews, then how do they assess the quality of the games they purchase? Well, they probably make an assumption based on the average game quality in the market...

If we apply the "market for lemons" reasoning to games, we can conclude that if the supply of games surpasses the reviewers ability to review them and the quality of the games falls at the same time, the result can be a downward spiral of falling average games' quality and prices. There will always be some exceptions to this overall trend. Developers who are able to signal their game's quality somehow - either by getting good reviews from recognizable reviewers, by advertising, having a recognizable brand or being featured by the store - will still be able to make decent money. However those who are unable to afford such signaling will suffer.

The Case of the Mobile App Stores

I believe we have already seen this play out in the mobile app stores - when iPhone and (more importantly) iPhone 3G debuted there were many high quality, innovative, payed games in the App Store. Many considered it an indie heaven and several well known indie studios' made their first success stories there. But pretty soon the influx of new developers both overwhelmed the reviewers and caused the prices to fall. And they kept falling, along with the average quality of games till they reached the minimum allowed price of $0.99. Soon even this barrier was broken with the move towards the F2P model, which has a ton of problems of its own.

Notice those the most affected by this situation are not the even the customers themselves. They still can buy some quality releases for the iPhone, either from large companies that can afford advertisements, branding, features etc. or from smaller developers with established brands (such as Vlambeer). In other worlds from those, who can afford to signal the quality of their games. The most affected group are the new indie developers, who can not afford traditional marketing and do not have enough brand recognition to efficiently use the social media channels. New indie developers are usually advised to completely avoid the App Store or to be prepared for a long brand awareness building process. Similar thing (only faster) happened in the Android marketplace.

The Case of Steam

I think it will not surprise anyone if I say that the next lemon market might be the Steam based PC indie market. Most of the indie games on Steam do not have Metacritic scores due to either not having reviews at all or having them from too minor sites. Steam allows developers to simply hide unfavorable user reviews from the store page, which reduces their utility. We have clearly seen the dip in average quality due to Valve opening their doors to the flood of old games that were too bad to get on Steam when they were first released, shovelware and even barely functional shit like the already mentioned Air Control. We have definitely seen a fall in incomes - the indie mantra of recent months was "getting on Steam no longer makes you rich". The question is whether this fall in incomes is causing a further fall in the average game quality, thus completing the cycle?

So far I have not yet noticed that. The optimistic explanation is that perhaps the incomes have fallen, but they are still at a level that allows developers to make quality games. The bitter-sweet explanation is that maybe the developers are feeling the sting, but they have no better market to take their games to, so they are biting the bullet and doing the best they can despite lower incomes. The pessimistic explanations is that the longer production cycle of PC indie games (compared to mobile games) means that we still have to wait for this effect to come into force. I have no data to prove or deny any of those hypotheses. I would like to believe that the optimistic one is true however, if the flood of shovelware continues I do not see a reason why the PC market should avoid the fate that has befallen the mobile app stores.

What Can Be Done?

That is the question. There are several things that can be done by Valve. The problem is that they will most likely not do any of them or drag their feet about it. Unfortunately I can only think of one thing that can be done by the developers.

The first thing that Valve can do is of course introducing some sort of quality control. This could even be profitable for Valve if this was done like the certification on consoles, where it is the developers or publishers that pay for the tests. However such a process would prevent Steam from releasing 200 games in a single batch, so I guess it will not happen.

The second thing, essentially a band aid, would be to at least change the store page layout to prominently show the game's Metascore and the user score - these are not perfect, but at least they could filter out the worst cases of shovelware.

The third idea would be for Steam to change their refund policy. Currently, as far as I know, the official policy is "no refunds" however they do grant refunds on a case by case basis. If this was changed in such a way that the players could request refunds for a week after a purchase, this would make the players more willing to buy the more expensive, quality indie titles according to the logic "if I do not like it, I will just request a refund". In fact this could even be an option enabled by the developer, as a signaling behavior to show the players "I am so confident in my game I am sure you will not want a refund". I am rather skeptical whether Valve would like that solution due to the specifics of the economics of microtransactions, but refunds are a well known and tested method of battling the information asymmetry problem.

The fourth idea is rather controversial. Since the cause of the problem is lack of reviews or users not reading the reviews Steam could introduce a rule where the only games accepted into the marketplace would be the ones with reviews. These reviews could be from some minor site or blog, so as not to reduce the chances for the indie developers and Valve would keep a database of trustworthy reviewers and perhaps even introduce their own Metascore. The scores and excerpts from the reviews would be placed in a prominent place on the store page. Of course I realize this solution would put a lot of power and responsibility in the hands of game reviewers and this might not be to everyone's liking.

The fifth idea is an idea for the developers - we could try reviving the old tradition of game demos. If the signaling is the name of the game, than "I will let you play a piece of the game for free, I am sure you will get hooked" is certainly a good signal to send. Of course this idea could also be helped by Valve cooperation if they somehow marked the games with available demos in the Steam store.

Conclusion

We, the developers, must fight the downward spiral of falling quality and prices. We must fight not only by making quality games, but also by calling for solutions that protect our customers from the lemon peddlers. Otherwise we risk the destruction of our currently best marketplace and with no obvious alternatives (no, not everyone can get on PS4) this could spell troubles for the whole indie game movement.

P. S. If you find the idea of a market failure interesting or if you find it preposterous and impossible, I highly recommend reading How Markets Fail: The Logic of Economic Calamities by John Cassidy. It is a very eye opening book and this is where I have originally learned about the market for lemons.

The article was originally posted on my private development blog at tryglawstudio.eu.


Related Jobs

Digital Extremes
Digital Extremes — London, Ontario, Canada
[10.25.14]

Sound Designer
Disruptor Beam, Inc.
Disruptor Beam, Inc. — Framingham, Massachusetts, United States
[10.25.14]

Lead 3D Artist
Red 5 Studios
Red 5 Studios — Orange County, California, United States
[10.24.14]

Graphics Programmer
Red 5 Studios
Red 5 Studios — Orange County, California, United States
[10.24.14]

Gameplay Programmer






Comments


Michael Joseph
profile image
Buying a lemon is also a great way to learn how not to buy one in the future. "Buyer beware" is imparting wisdom.

The problem with the type of quality control you're talking about is it encourages people to trust the provider of QC rather than depend on themselves. That's fine for things like construction work, air travel, car manufacturing, food production, etc. But for cheap games? Fool me twice shame on me should suffice.

And haven't we essentially already come from that direction with the old boxed retail gate keeper system and which contributed to a stagnant industry? Let's try the other way.

A market full of crap shouldn't be a threat to people who are genuinely talented and are making good software.

Tomasz Mazurek
profile image
Well, the key point of the problem is that it might happen that customers will not apply "fool me twice" rule to individual developers, but to the indie games as a whole. And then the market full of crap does become a problem even to people who are genuinely talented.

Maria Jayne
profile image
As much as I favor the "tough love" approach to users buying bad games and learning from their mistakes. The real fear is that there will be so much crap that even if a good game is under it all, nobody will bother to find out.

Which will perpetuate the issue, if you can't get recognized as better than average, you may as well be average and save yourself some money.

Pete Devlin
profile image
I have to be honest and say this comes back to the developer who must fight to find their audience and keep them. If your peddling lemons you are certainly not doing that. Sure, your devaluing the market for other developers, but your not having any impact on those who are building genuine audiences. Edmund McMillen, Simogo, slow burners building big audience who anticipate their work. The problem is indie developers bitching about not being 'found'.

'How does anyone find quality amongst all the crap' is the basic chant. 'Finding' is the keyword. Like many a reality TV show star they are waiting to be 'found'.

I love the idea of a diverse marketplace. It allows a vast range of product that can suit many a niche so I'm completely OK with the store policy. If the developer works hard to get the message to me then they've earned the sale.

By the way, interesting article, worth the read to learn about the lemon model of economics.

Lipton IceTea
profile image
Tomasz Mazurek stated in his article. Developers with brand awareness / marketing budget or review will always find ways to get attention. Those affected will be the new comer with low marketing force. Customer will judge their product base on the overall game quality provided by the plateform. They might have done a great game, but if it didnt create any word to mouth dynamic and dont have the funds to push the marketing, then they will go down to oblivion aside shovelware.

Simon Ludgate
profile image
I think the simplest solution is if Steam implements a mandatory timed demo for every product using some sort of price-to-time formula, with game sellers permitted to offer extended demos if they desire (either increasing the base demo time or using "free weekend" style promotions). Some base formula like 5 seconds of free demo time per cent the game costs.

So a dollar game means you can try it out for a little over 8 minutes, and a big $60 blockbuster gives you a little over 8 hours of trial.

Although that solve the initial "I can immediately tell this is crap once I start playing it" problem, it doesn't stop the technical "lemon" problem of discovering the game is bad after it's too late (what if you don't realize the blockbuster is crap until the 10th hour?), however implausible that is.

It also fails to sole the much more serious problem of discovery: potential customers still have to find and download the trial.

I think good curation / quality control has as much to do with selecting good content so the viewer's choices are simplified as they are about ensuring that bad content doesn't show up at all. I know my own spending habits on Steam have dropped significantly since the floodgates of crapware have opened on the site. I used to regularly check the "new games" list to find things to buy; now I notice it defaults to "top sellers" instead and I basically don't bother checking at all any more.

Thomas Henshell
profile image
Great article, and very interesting connection to the used car market.

Matt Robb
profile image
I believe their user stores or whatever it is they're calling it exactly is intended to help with and/or solve this. Basically, they're offloading their curation to the community. If you can find one of those people who tend to agree with you on games you already have, you can generally trust their opinions on games you don't have. Basically combining reviews and curation into one construct.

Curtiss Murphy
profile image
The 'store' curation idea seems pretty sound. Find a 'store' promoter you like and buy the games they recommend. The science is that you'll be happier, even if the game is crap... We like to do what our friends do.

Shervin R Rad
profile image
Valve thinks customers are wise and I think it's really great because they are.
They can't use traditional quality assurance anymore because it's slow so instead they have made some interesting tools for the community to do this.
User reviews are so much useful now. In addition their vision to give the users the opportunity to create their own markets will help this crowd quality assurance to happen.


none
 
Comment: