Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
July 25, 2014
arrowPress Releases
July 25, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Creative Assembly's 'fairly brutal' approach to achieving high Metacritic scores
Creative Assembly's 'fairly brutal' approach to achieving high Metacritic scores
April 30, 2012 | By Staff

April 30, 2012 | By Staff
Comments
    19 comments
More:



Total War developer Creative Assembly shoots for a 90 percent Metacritic score for its games, and has a "brutal" approach to getting there, the studio tells Gamasutra in our latest feature.

Metacritic's average review scores have become increasingly important for developers in recent years, as many publishers and consumers refer to those numbers as an indicator of a game's quality, as well as the quality of a studio's output. In some cases, those scores can determine whether developers receive bonus payments from publishers.

Creative Assembly's studio director Tim Heaton says that his team goes through a rigorous process when deciding what features to include in a game. The Sega subsidiary will cut some features immediately, but it will prototype and judge the quality of others before deciding to ditch them. And while prototyping ideas and examining their process, the group will also abandon features that it feels will take too long to turn into a high-quality addition.

"Through production, actually, we do what we call 'Metacritic analysis,'" says Heaton. "We will break those features down into subsets, and we both look at it from a player's point of view, and a reviewer's point of view, and we'll weigh certain features as to how we see players and reviewers look at them, and they'll build up to a 100 percent score, and then we'll judge where we feel we are on those individual feature sets, and see the momentum on those and the velocity on those, too."

He adds, "And so if we see one flat line and it's not where we want it to be, we then will cut it. Well, we'll cut it really late in the day. I think teams are really scared about doing 90 percent of the work and then cutting it. It's kind of like, 'Well, it's nearly finished; I... I've done all the work! Please don't cut it! I'm sure I can make it better.' And we're fairly brutal on that."

Heaton says he would much rather reject a feature that the studio invested resources in, than have it left in the game and affecting its quality. "You know, every step of the way -- from the beginning to the end -- we're talking about a 90 percent Metacritic," the Creative Assembly director emphasizes. "That's our goal. That's what we tell Sega. And we communicate that through graphs, basically, of where we think we are."

He continues, "We build into that also, on that Metacritic analysis, external events. So if we think we've done a really great PR job, if there's an individual event that we've done really good, we might add, you know, a .5 percent Metacritic. If we think it's fucked up or somebody's not done their job right, or miscommunicated something, or whatever, we'll see that in our Metacritic analysis. And we share that with Sega on a weekly basis, so that they can figure out how we're doing, too."

Heaton admits that cutting a feature that might have been nearly completed can affect the studio's morale, but he believes the team eventually buys into the idea that these decisions are for the best: "Certainly some elements of a team -- and this always happens with every team I've ever worked with -- just go, 'Come on, just give me a game design document. Just tell me what I need to do and then I will do it to the best of my ability.' And we slowly, hopefully, educate people that's just not the best way.

"And we will enter a fog of ideas for quite a long time, and some of those things will have risks against them right up until the end, and then we might pull them. It pisses people off, absolutely, but it's for the best. But nothing makes the team prouder than delivering a 90 percent game and selling two million copies. So that's the bottom line, and people do come to understand that."

The full feature interview, in which Heaton talks more about Creative Assembly's process for building high-quality games, is live now on Gamasutra.


Related Jobs

Activision Publishing
Activision Publishing — Santa Monica, California, United States
[07.25.14]

Senior Online Programmer - Central Tech Online
Activision Publishing
Activision Publishing — Santa Monica, California, United States
[07.25.14]

Online Programmer - Central Tech Online
Raven Software / Activision
Raven Software / Activision — Santa Monica, California, United States
[07.25.14]

Technical Director - Central Tech Online
Vicarious Visions / Activision
Vicarious Visions / Activision — Albany, New York, United States
[07.25.14]

Environment Artist-Vicarious Visions










Comments


Maria Jayne
profile image
I'm pretty surprised how much emphasis publishers and developers put on metacritic. Somebody made a review site of review sites and now that drives the industry and decision on bonus rewards apparently.

Somebody should make a site were only customers can vote on the product instead of reviewers. I mean actual purchasers of the game, not random internet sock puppet that wants to neg vote a game for whatever childish reason.

Bonuses should be applied based on what customers think, not reviewers, they didn't pay a dime.

Benjamin Quintero
profile image
I think this wouldn't work because you'd end up with 90% of your scores split between 1's and perfect 10's, with a modest few in the middle. Gamer critiques tend to look more like a cereal bowl than a bell curve. In general, if you like a game you will quietly enjoy it. If you LOVE/HATE a game, you run to the internet to scream about it. =(

Stansilav Simovski
profile image
This is easily solved with having a Youtube-like rating system. It's binary and you just look at the ratio. Very good for crowd-rating.

[User Banned]
profile image
This user violated Gamasutra’s Comment Guidelines and has been banned.

Maria Jayne
profile image
It's a fair point, about happy people generaly not making any noise at all, however if you have an equal ammount of raging angry customers and extatic ones they effectively cancel each other out. If you game has a bias one way or the other that would be telling.

A lot of games now want you to register an email address, sign up for an account etc, I don't see why when people register a new game for these accounts the developer/publisher couldn't email them a questionaire on how they enjoyed the game 3+ months after they bought it, I say 3 months partly to give angry players time to get perspective and partly to give everyone regardless of circumstance time to get time playing the game.

Granted some people will use fake, unchecked email addresses but if they know they will be asked what they thought of a game it may even encourage them to give real contact details.

Dario, I wasn't suggesting you have no right to an opinion, I was suggesting an opinion from a consumer is a more valid source for judging a companies quality or product.

Gary LaRochelle
profile image
Ah, for the good old days. When all you had to do to get a good review score was to slip the reviewer a few bucks (or buy ads on their web site).

Jeremy Parsons
profile image
This doesn't sit right with me at all.

*I had more to say, and I actually typed it all out, but I cut it before hitting submit.

Shea Rutsatz
profile image
I agree. It doesn't really seem like a healthy environment, or much fun. Basing your projects on Metacritic is silly, and I wouldn't be terribly surprised if they have a high turnover rate.

Jeremy Parsons
profile image
Yeah, I don't think I'd be willing to stick around all that long if I worked for months or years on something and was told the game would be better without anything I've ever touched, especially using "anticipated metacritic scores" as the measure for "better."

I wonder how long he's implemented this development strategy, since they only seem to have 2 metacritic 90s in the past 7 years, out of 28 possible listings in that time frame (and those 90s have user ratings of 8.1 and 6.8).

Joseph Arcidiacono
profile image
Unfortunately, Creative Assembly implies that making difficult decisions (eg cutting features late in development) and making a game based on what a reviewer thinks are equally important actions. While there are many ways to make a great video game, personally I don't believe in the appease-the-critic method.

Vin St John
profile image
It's interesting because in many cases, those 'no completely there yet' features shouldn't be dragging down the rest of the game just by being included with it. A tacked on multiplayer component is usually better than no multiplayer component at all, once the work has already been done.

It seems to me like in these cases, if the dev is worried that including the 'unpolished' feature would hurt their initial reviewer/fan reaction, they should just keep working on it and release it in an update 6 months later - when decisions have already been made, and it will be seen as icing on the cake instead of a piece of an unfinished product.

Patrick Haslow
profile image
I'm sorry, but I simply find a great deal of dubious credential amongst the so-called 'videogame press'. It really saddens me that so much emphasis is placed on its opinions by both the financial and creative sides of our industry. Anyone remember this?:

http://arstechnica.com/gaming/news/2010/11/call-of-duty-black-ops
-review-event-press-gifts-detailed.ars

Patrick Haslow
profile image
I like to think/hope Kickstarter is something of a big kiss off to this line of thinking.

Jack Lee
profile image
It weirds me out how often he mentions Metacritic specifically and by name, but I think he would have a solid idea if he just reworded it to say "We want every part of our game to be excellent, and if it's not living up to our quality standard, we have to cut it." The idea of excising parts of your game that don't make it better isn't a bad one, it just seems a shame that their reasoning behind this practice seems to stem solely from a need to get high Metacritic scores and impress reviewers. I'm not so idealistic to think that those things aren't important; history has proven that while marketing and ad budgets can help a game in initial sales, word of mouth and demonstrable quality can really make a franchise. I guess I just hope that game creators can also value their creations as works of art and culture as well as consumer products, rather than be wholly consumed by market forces.

Nicholas Bellerophon
profile image
I think some of us, and perhaps the article also to some degree, are glossing over some details. But note:

"we both look at it from a player's point of view, and a reviewer's point of view"

So really it seems to me what they're doing is not completely insane. I mean, who do you trust when it comes to deciding whether a game is going to be good or not? And moreover, is it not fair to say that a lot of the marketing of a game can come down to PR in the gaming press? So yes, there is the target audience of players, but also a kind of 'intermediary' target audience of press, who will market a good game to players for you.

Now, the revenue model is also very important. For CA games, you have to pay upfront, so actually the customer needs to make a decision before really getting into the game. So press is, again, very important, though admittedly word of mouth is also very important. But I think CA recognize that.

Gil Jaysmith
profile image
I happen to know that many people who were at CA when I was working there from 2001 to 2005 are still there in 2012. Asserting that it can't be much fun and people must leave in droves is uncalled for, to put it mildly. (Anyone in this thread who's a student, for example, kindly shut up about game production until you've done any.)

Jerome Grasdijk
profile image
Calling this design-according-to-the-reviewer is rather missing the point. After all, it is done before review scores are known, and it's nigh on impossible to produce a valid breakdown of review-score-contribution by game feature from a real review anyway.

But canning unfinished or half finished work late in development is nothing new, surely. Anyone who has shipped a few games will have seen this. As Gil says, when we were there the staff turnover was not huge, on the contrary there were many long-timers.

Matt Cratty
profile image
This perfectly highlights the broken thinking that is steering us deeper into the gaming dark ages.

Call it rhetoric all you like.

Dany Rioux
profile image
Goes to say, studio execs aren't brain powered.

If you want to know if your features are good, don't do internal reviews based on nothing. First you're biased, second metacritic is, and always has been, broken and mostly irrelevant.

If you want to know if the features are good and worthy to gamers, ask gamers to rate them. Do it the way Valve does it. Get internal testers, pay them, ask their advice, what they like, don't like with what you did, etc.

In short, use your brain, it's not that hard!

THIS is why we can't have nice and innovative games anymore. -__-


none
 
Comment: