Gamasutra: The Art & Business of Making Gamesspacer
Storyboarding for Games User Research
View All     RSS
September 1, 2014
arrowPress Releases
September 1, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Storyboarding for Games User Research

February 13, 2013 Article Start Page 1 of 3 Next
 

The quality of video game design ultimately depends on a developer's creative vision. However, evaluating how close the final game experience is to the original developer's intent is often hard. Games User Research (GUR) is an emerging field that aims to get the product closer to the developer's intention in terms of the player experience. GUR analyzes the interaction between players and games to get insights from players to improve game design (see here for a Gamasutra feature on what GUR is).

Games user researchers (GURs) use many different methods, tools and techniques to gather information about players and their gameplay experience. This information is used to support design decisions of game developers (see here for another Gamasutra feature on GUR methods).

When conducting user test sessions (either internally or through contractors), applying correct methods and thorough data analysis does not improve anything if the final GUR results are (1) not communicated well, (2) not convincing, or (3) not actionable for the development team.

User Research Reporting

In this feature we are focusing on a little-explored area of GUR: reporting. We will be looking at how user test (UT) findings are communicated among game development teams and how game development can benefit from storytelling and storyboarding techniques (which are common in web development).

We interviewed six game development professionals from midsize UK game design studios to explore storyboarding. We wanted to know how they communicate UT findings, what are the limitations in current approaches, and how these approaches can be improved.

For our developers, the main values of UT sessions are to see: (1) areas of frustration, (2) areas that are difficult to pass (blockers), (3) if the players are having fun, and (4) if players understand the game and are using all the game features. They mentioned that an ideal report would be a process to capture a massive UT data and report it in a way that it is easy to make sense of.

For example, in a previous title worked on by one of the developers (a racing game) they collected game metrics to generate a crash heatmap of each track. They added: "from heatmaps we could see the crashes, but we know they can lead to different experiences. Some of them lead to enjoyment and some lead to frustration; the heatmaps won't show this difference [...] [an ideal report] is somewhere between only seeing the heatmaps and talking to the actual players." They suggested: "for some issues you wouldn't feel a text report could put them in a right context and time line. For example, when interpreting from a text report, there is no way to see the change of pace and enjoyment."

Let's summarize the most important aspects of UT reports identified from these interviews:

The report summary is the section they all read and found most useful. All of our developers think a good UT report should show an at-a-glance-summary of a level.

Location of issues in each level is an important factor for prioritizing fixes for them. Our developers want to see where exactly the issues occurred. For example, they said the "UT report should show me where my good and worst parts are, this can help me to prioritize what to fix. It is more concerning if a negative experience is happening at the beginning of the game, because people can easily drop out there."

Trust is a vital matter for UT reports; if the development team does not trust the results the problems will stay. Our interviews suggested that the most convincing case is when the developers personally attend UT sessions and have a face-to-face conversation with players or watch the gameplay video. They also said "to trust a UT report, it should provide evidence of why something is wrong with the game; only stating the problems won't be enough."

Our developers want UT reports to enable them to make a comparison between the player's experience and the experience they intended to design for. They think it would be great if a report could show an accurate match to their intentional design. They suggested: "if using the UT report as a comparison tool, it has to distinguish between usability, user experience, and pace in the game." For example, the existence of a usability issue would not be intended, and it is not a way for pacing games.


Article Start Page 1 of 3 Next

Related Jobs

Zindagi Games
Zindagi Games — Camarillo, California, United States
[09.01.14]

Software Engineer
Zindagi Games
Zindagi Games — Camarillo, California, United States
[09.01.14]

Lead/Senior Designer
N-Fusion Interactive
N-Fusion Interactive — Manalapan, New Jersey, United States
[09.01.14]

Unity 3D Game Designer
Retro Studios - Nintendo
Retro Studios - Nintendo — Austin, Texas, United States
[09.01.14]

RETRO STUDIOS - Level 3 Engineer






Comments


Dennis Kappen
profile image
Great article!!. This is an exciting area of research and has immense potential.

Typically a storyboard is a representational process of visually depicting the structure of a story. A sequence of visuals helps to communicate the progression or build up of the story. Storyboarding has always been used by designers and concept artists to flush out their ideas in the ideation phase of a project in the fields of industrial design, multimedia, animation, web development, game design and films to mention a few. The storyboarding activity is a means to create a tangible expression of an intangible collection of thoughts by a designer or a team of designers.

In my mind, storyboarding is a quick, iterative and a visual representation of a collection of ideas, which together communicates the overarching schema or the design intent of a proposed solution. The visual representation can be in the form of sketches, flowcharts, or mind mapping bubble diagrams on a simplistic level. In the process of design, the storyboarding process has been a brainstorming method whereby expensive production costs can be saved by investing time in storyboarding concept designs in various phases of any project in the above mentioned fields.

Hence, in storyboarding, all one needs is an idea which is then transcribed into sequential visuals, which then communicates the essence of the scenarios being conceptualized or designed. This activity has been used in all the three stages of the concept design process namely; ideation (the creative and brainstorming phase); design creative (act of transferring concept ideation into tangible visual elements); and design validation (testing the functionality of the design being conceptualized to ensure that the creative solution adheres to the goals set out in the design brief). The concept design phase is then followed by an iterative loop interconnecting future phases; design and development; prototyping; validation and testing.

My point here is that, semantically, storyboarding is an idea representational tool or a scenario representational tool in the process of concept development.

In the above article, the term “storyboarding “has been used by the authors’ as a means of representation of data visually. The question that bothers me is that if the data generated in the UT is already a tangible element, is it still appropriate to use the term storyboarding to visualize data? If the data is being represented visually, then, is the representation semantically, a data visualization interface as opposed to being a storyboard? In my humble opinion, the term storyboard in “Biometric Storyboards” may not be appropriate in its usage, because the representation is of finite and defined data as opposed to ideas. Perhaps, in my mind, the term “storyboards” adds confusion between idea representation; and representation of data or multiple permutations and combinations of data visualizations.

On the other hand, if Biometric Storyboard as a tool would help designers to build “what if” scenarios from the collected data to project or simulate future responses, then it lends itself to being a concept/scenario generation tool as opposed to being a data visualization tool.

Simulations of new scenarios based on mapping between the intended, the perceptual and the actual interactions would create a better understanding of users’ needs and interaction challenges.

In my humble opinion, the question we need to ask is while a storyboard is a visual representation, is every data representation a "storyboard"?

Lennart Nacke
profile image
Dennis, this is a great question that we actually thought about close to the end of the development of Biometric Storyboards. Really, what is being generated by the user research and biometric data is a visualization of game structure, much like a plot graph of important game events. So, following your definition of storyboarding as an ideation tool, I guess we are closer to a filtering tool than an ideation tool. I think part of the understanding that we still have to gather is whether storyboarding itself is a useful term for this kind of data representation or whether there could be better terms for quick insight tools that visualize big data for gaming. I think storyboarding and ideation is where our approach started, but it would be interesting if there are more appropriate terms for this as we go toward visual data analytics.

Rina Wehbe
profile image
The article was well written and clearly outlined a problem. Biometric storyboards were purposed to help game design teams communicate ideas effectively by using physiological inputs to get information about the user’s emotional state. However, physiological methods are proficient at determining the valence of the arousal. Is gameplay behaviour enough to determine the valence of the arousal? Could you further talk to this point? I would also be interested in hearing more about you ideas concerning using psychographics to determine motivations and patterns of behaviours. Overall great article!

Pejman Mirza-Babaei
profile image
Hi Rina, glad that you liked the article, also interesting thoughts and questions. I don't think we are that advanced to be able to determine user's emotional state yet. In Biometric storyboards , the graph represent user's level of arousal (excitement or frustration) and qualitative data (post-gameplay interview) are used to determine their valence. These player experience information would be more useful when compared to designer's intended experience. Also, I don't think gameplay behaviour is enough to determine player experience (see the heatmap example in the first page), and that is why GURs use triangulation of methods. Also have a look at this article regarding your question about using psychographics to determine motivations and patterns of behaviours http://www.edge-online.com/features/biometrics-future-videogames/

Dennis Kappen
profile image
Thank you Lennart, not getting bogged down by semantics, this is a great article and I am also reading the CHI2013 paper to understand interrelationships between player experience, intended, perceptual and actual interactions and get more clarifications on physiological methods.


none
 
Comment: