Gamasutra: The Art & Business of Making Gamesspacer
Waterfall Game Development Done Right
View All     RSS
September 22, 2014
arrowPress Releases
September 22, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Waterfall Game Development Done Right

November 21, 2012 Article Start Page 1 of 3 Next
 

Agile, Agile, Agile. It's hard to disagree with a project management process named "Agile". It's like disagreeing with something named the "Patriot Act" or answering "No" to a department store checkout attendant that asks if you would like to save 20 percent on today's purchase. It's clear: Agile is cool, and every Scrum master certificate mill will agree with that statement.

But what happens when an Agile paradigm isn't appropriate?

"How is that possible?" says the guy with the skinny jeans and the PBR.

There are two scenarios where we think a more "waterfally" approach is appropriate (We'll explain the "more" part later... because no one could ever be purely waterfall if you think the name implies a rigid schedule).

The first is when a project is innately low-risk, and the second is when your stakeholder doesn't wholeheartedly buy into the Agile process. Examples of the second include Kickstarter and the majority of contract work.

For the first case, I think you can throw out any game development scenario where you are striving to achieve fun that isn't a full clone of something that is already proven to be fun. Fun is just too damn squishy, and in my experience, you don't know something is fun until you play it and say, "Hey, that's fun!"

Creating fun is a risky proposition. Vertical slices -- a technique where you make some small percentage of the game at near final quality -- and rapid prototypes help pave the way for a more structured project.

The second case is the focus on the remainder of this article. GarageGames is a service company and in the majority of the cases we see, the formula for customer's desire on an initial consultation is usually something similar the following:

I want X by the date of Y. How much will you charge me?

Problem Statement:

A customer wants a firm fixed price (FFP) quote for a game-related software development project. How does a company arrive at a price and date that is fair for the procuring and commissioned company?

To simplify the problem statement, what the customer wants to know is: What, How, When, and How Much is required. Additionally, you will need to determine Who from your firm will do the work. Those elements, solved together are a FFP contract, schedule, and Statement of Work (SOW). Here are the steps we use to build them:

  1. Determine initial scope -- Focus on the What.
  2. Determine if the project is suitable for FFP contracting -- Initial analysis of How.
  3. Build a rough order magnitude estimate -- Second analysis of How, first analysis of When and How Much.
  4. Determine scope of the Statement of Work -- Final analysis of What.
  5. Determine & deliver final quote -- Final analysis of How, When, and How Much and Who.

Determine Initial Scope

This usually occurs through one or more conversations with business development, technical, and project management. If a GarageGames employee can do all three it can often be covered in one conversation. The most important goal is to be output and goal focused. Don't get too caught up in implementation. Right now you are more interested in the what than the how or when. For us, this process of determining initial scope starts with a conversation about the customer's high-level goals.

Often times, the customer doesn't know what they want aside from their high-level goals; sometimes, you need to help them understand their goals (a bit of a red flag sometimes). It's understandable that they might not know; if they really understood the solution to their high-level goals then they wouldn't be coming to you now would they? Work with the customer to define and document the high-level goals. By understanding them, you are in a better position to either develop their plan or validate any existing plans.

Knowing how much can be helpful at this stage. Customers typically don't want to tell you their budget at this point because they are worried that you won't give them an honest quote --you have an asymmetric knowledge advantage that makes them uncomfortable. A customer's budget holds intrinsic clues about what can be accomplished and not knowing that number will make the process much harder to scope and define the project.

GarageGames asks customers to try and identify a budget range for the purposes of accelerating the quote process. We assure them transparency in our quote that addresses their fear and we've found that sharing more detail with the customer gives them a better experience. Transparency puts both parties on the same side of the table tackling a problem instead of negotiating against each other as foes. Working together to bring down cost is something we enjoy doing. We believe that if we save customer money they are more likely to spend that money on us in the future. Both sides win.

If they aren't willing or allowed to share a budget range with you hope isn't lost -- it's just more work to triangulate the cost from the bottom up.

Determine if the project is suitable for FFP contracting

At this stage, we are going to analyze the project by doing an initial analysis of how we will implement the project to meet the customer's high-level goals.

It's time to start thinking about risk. All projects, especially software projects will have some risk. How confident are you about your team's ability to accomplish this project? What is the variance of success and failure? At GarageGames we measure risk using a confidence scale of 1 to 10.

With an eye towards risk management, we do our initial analysis of how we will implement the solution. To do this, we work with technical leads to formulate an approach.

We decompose the output above from high-level outputs into smaller features. This process is known by many as a Work Breakdown Structure (WBS). At GarageGames, we tend to get to a more detailed decomposition than many. It takes more work, but it protects us from taking a job we can't complete by exposing more known issues.

For each of the features, we have the leads describe the expected implementations. How will you develop this feature? How confident are you that this approach will work? Document the confidence and determine a gut feeling about your confidence in the implementation of the entire project. A task estimate may look like this:

Implement Frustum Culling   Estimate: 20 hours   Confidence: 7

One of the biggest mistakes made by companies is saying "yes" to a job they can't perform. If your gut feeling is that you aren't very confident in your ability to accomplish the work, please do yourself a favor and either 1) walk away or 2) insist on a time and materials contract where you are paid hourly for the work as you accomplish. Saying yes to a FFP job you aren't confident in will likely hurt your business as bad as your prospective client. Just say no!

If you are having challenges decomposing the steps or the cumulative confidence numbers are low, this might be a bad job for your company.

GarageGames Confidence Scale

Below is the scale GarageGames uses as a standard for confidence numbers:

  1. No clue -- don't make decisions on this. We need to talk more.
  2. Hardly a clue -- don't make decisions on this. We need to talk more.
  3. Someone else has done this and I read about it; job not well defined -- don't make decisions on this. We need to talk more.
  4. Someone else has done this and I think I understand this; job might not be well-defined -- don't make decisions on this. We need to talk more.
  5. Done something similar to this before and this relates to that work -- this estimate has a variance of +/- 50 percent of estimate.
  6. I think I understand this fairly well and I understand the goal.
  7. The average case for programming when the requirements are understood.
  8. A confident case for programming; average case for a lot of art work.
  9. I've done this before and it's very similar.
  10. I'm probably not working on software... or... no matter what; I'm only going to work on this for a period of time (i.e. 1 week of QA).

Article Start Page 1 of 3 Next

Related Jobs

Infinity Ward / Activision
Infinity Ward / Activision — Woodland Hills, California, United States
[09.20.14]

Producer - Infinity Ward
Infinity Ward / Activision
Infinity Ward / Activision — Woodland Hills, California, United States
[09.20.14]

Senior AI Engineer
Infinity Ward / Activision
Infinity Ward / Activision — Woodland Hills, California, United States
[09.20.14]

Lead Tools Engineer - Infinity Ward
Infinity Ward / Activision
Infinity Ward / Activision — Woodland Hills, California, United States
[09.20.14]

Senior Tools Engineer - Infinity Ward






Comments


Eric Preisz
profile image
I don't think I would advocate getting paid per day (perhaps there is some confusing text in the article). We build estimates on the granularity of a day. Payments tend to be on delivery/acceptance of milestones.

Clinton Keith
profile image
I fundamentally agree with the Eric on the gist of this: why employ purely iterative and incremental development approaches when there is little uncertainty.

However, a problem common with comparing agile and waterfall is the labeling. Few, if any, studios are purely "agile" or "waterfall". They are more mindsets that encompass a wide variety of practices and approaches to development. Labels are convenient for helping make an argument, often with cute little straw-man statements to help reinforce preconceived notions.

In reality, its more of a scale from pure iterative, inspect and adapt practices to assembly-line approaches that need little inspection. Studios have *always* been somewhere in the middle of this scale. Individual practices are employed based on uncertainty. To make the argument that "waterfall works" fails here. Sure, *some* waterfall practices work when we know more up front. You might say that you don't need any agile practices, but that's like saying you'll never need to inspect what your making and how your making it...not a state that any game developer has been in for very long, in my experience.

Anyways, I love the confidence scale. I'd love to try doing this on a project with some leads using planning poker! Thanks for writing this Eric.

Eric Preisz
profile image
"However, a problem common with comparing agile and waterfall is the labeling. Few, if any, studios are purely "agile" or "waterfall". They are more mindsets that encompass a wide variety of practices and approaches to development."

Well said.

However, I do think that the popularity of Agile methods has somehow colored developers and diminished the discipline and acceptance of prior planning when possible. The pendulum has swung too far in the belief that all development require a reactive only project management process.

Clinton Keith
profile image
Agreed Eric. I call those projects "Incremental and iterative death marches" ;)

Chris Toepker
profile image
Eric, the confidence scale is a thing of beauty, putting into words things I think every day. Thanks for that!

As for Waterfall, what I think is interesting is how we (as the 'new industry'?) seem to want to invent new words all the time. I've found working in manufacturing the approach is just called "project management." Not as sexy a name perhaps, but when done fully and professionally its about about as waterfally as you can get, covering the scoping and quoting and resourcing and etc. questions quite well.

And when an "agile" approach is needed, a design phase is added. In other words, you don't commit all the resources until you really know what you want to make...whether it is some fun (in our case) or stylish (every bit as squishy, no?!) or ergonomic or what-have-you. Don't know what I mean? Check out: www.pmi.org/, especially: http://www.pmi.org/PMBOK-Guide-and-Standards.aspx

James Hofmann
profile image
I think the essential difference between manufacturing and software is that hardware's physical constraints and lead times produce a lot of known-quantity requirements and deadlines that aren't negotiable, while software allows everything to be negotiated, so experienced and well-managed practitioners can preserve "any" amount of desired flexibility into the end of the project, at the cost of weighing down development of individual features.

Tom Kofoed
profile image
Dont blame Water Fall or Agile/Scrum
Blame the the person who use a method incorrectly or to religiously (Rigid).....
:)

Eric Preisz
profile image
@Chris - Originally, I used a 1-10 scale of "risk" instead of "confidence". I found over time, it was easier to get people to give more consistent results using the later. Collecting confidence is all about managing and quantifying risk. An estimate without a risk assessment is missing a lot of valuable detail.

Jeff Murray
profile image
Great article! Nice to see an alternative view on this subject, for a change.

Speaking from my experience, I've worked with both Agile and Waterfall with games programming teams between 4 and 13 programmers. Although we saw better engineered products with Agile, we ended up with better (more fun) games when we used a more Waterfall-based approach. Was that the methodology, the team or the game design? It's hard to say- these are organic products at the end of the day! Given the choice, I'd try to aim somewhere between the two and see if it is possible to balance them out.

We did risk assessment in a similar 1-10 scale, but found that without a strong PM on the project, they were difficult for clients to understand / deal with. Clients would still ignore them, turn tail later in the project and go ahead regardless!

Christopher Lambert
profile image
Over 18 years software development experience here, my feeling regarding process is quite simple - there is NO process until you understand the company, the project, the people and technology at play. Agility is about selling books for the most part, as is Waterfall. In my opinion you should be examining the situation and developing a strategy on a case by case basis and using aspects of all processes as a tool chest from which you piece together something to suit the needs of a given project/company/team/technology.

In my case, I find Boehm Spiral Model to still be the 'best general fit' after all these years and frankly see no advantage of waterfall or agile over this approach 'in the general case'.

Svein-Gunnar Johansen
profile image
I would argue the following: If there is a place for waterfall, it is in building the first iteration of your project: The "walking skeleton". And only if this can be built with high levels of confidence.

Robert Riedl
profile image
A third scenario to use 'waterfall' would be when you've created something very similar. In this case, you can use your actual time from the previous project to set up your schedule.

Speaking of actuals, I'm curious how accurate your risk modifiers were. Sometimes estimates can be too conservative and adding even more padding can unnecessarily increase your project cost estimate.

Richard Ranft
profile image
Having actually worked for Eric I'd like to say that as long as everyone is honest in their assessments of their tasks this seemed to me to be fairly close on the average. As always, we have to remember we're "estimating" and probably won't have a number that lands within minutes of the actual task duration.

Nickolas Anderson
profile image
This was a good read and very insightful. Thank you, Eric. The commentary surrounding it has been poignant as well.

In my opinion it is best to be pragmatic about methodology and to not become wedded to a single approach.


@Eric: The Confidence Measurement is fantastic! I'm going to have to try that.


@Christopher: The Spiral Model is an interesting visualization of the development process. I hadn't encountered it before. Thanks for sharing that.

Brian Dreyer
profile image
I’d defer to the “Agile Manifesto” or the 12 Agile Principles:

1. Does what we’re doing at this moment support the early and continuous delivery of valuable software?
2. Does our process welcome change and take advantage of change?
3. Does our process lead to and support the delivery of working software?
4. Are the developers and the product owner (stake holders) working together daily? Are customer and business stakeholders working closely with the project team?
5. Does our environment give the development team the support it needs to get the job done?
6. Are we communicating face-to-face more than through phone call and email?
7. Are we measuring progress by the amount of working software produced?
8. Can we maintain this pace indefinitely?
9. Do we support technical excellence and good design that allows for future changes?
10. Are we maximizing the amount of work not done- namely, doing as little as necessary to fulfill the goal of the project?
11. Is this development team self-organizing and self-managing? Does it have the freedom to succeed?
12. Are we reflecting at regular intervals and adjusting our behavior accordingly?

The big issue with Waterfall is change – the team/people, features, tech, etc., true Agile deals with this better, quicker, cheaper and allows you to kill what needs to be killed faster.

Agile, in my humble opinion will never work without a dedicated ScrumMaster and that’s not (always) a traditional Producer/PM – especially when there's Jr. people on the team.

Eric Preisz
profile image
@Brian - I don't see anything in the Agile Manifesto that we don't apply to our practices. Although, from my experience, I've seen #9 overrule #10 more often than it should...but that's another article.

Dan the gaming Guy
profile image
I like having a macro waterfall schedule that breaks project features / efforts up into buckets (this ensures your project is not a run away train of endless itteration), within each of those time buckets use an agile and fast itteration process to accomplish desired outcome of each part.

Also ensure to schedule a very healthy polish (around 25-35%) phase at the end of the project to:
1) Reduce risk of over scoping your product by having a shorter production phase
2) Create some buffer if youre budget is reduced, your asked to bring your dates in or if something else goes wrong with staffing/hiring/planning, etc
3) If you avoid the above scenarios, you should be able to create a highly polished product

"If you can make every aspect of your product good, your result will be great".


none
 
Comment: