Gamasutra: The Art & Business of Making Gamesspacer
The Case for Casual Biometrics
View All     RSS
August 1, 2014
arrowPress Releases
August 1, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
The Case for Casual Biometrics

December 20, 2012 Article Start Previous Page 3 of 4 Next
 

The Second Set of Biometric Experiments

A second series of tests was conducted two months after the initial experiments. The aim of this second series of tests was to understand how a player's performance develops during the first 10 minutes of gameplay.

An important point to clarify is that in the competitive version of the game, the walking speed of the paper creatures increases incrementally after a specific number of creatures have been correctly recognized. The idea behind this design parameter is that if our fantastic creatures are not properly fed, their actions become more frenzied as they grow hungrier in their search for food. In this phase of the testing, the development team wanted to find an ideal balance between the initial speed of the game and the rate of acceleration as players progressively gain skills during their advancement of the game. Specifically, we wanted to find answers to the following questions:

  • Does the game allow players in our target audience to be proficient enough to endure play-sessions of five minutes after three or four games?
  • Does the game make our players excited but not anxious?
  • Do players have a generally positive reaction to the reaching of the Game Over state (which needs to be perceived as fair and encouraging)?

Visual Results for the Second Set of Tests

The graphs below visualize the test results of one of the 14 test subjects that was tested in the second phase of experiments. Each dot on the graph represents a different beast. Blue dots represent beasts that were walking at basic speed, and red dots represent beasts that were wobbling faster than the basic speed.

In the test version of Gua-Le-Ni, the paper beasts were accelerated after the appearance of every four specimens. The vertical line on the graph represents a Game Over state, after which the game resets.


Skin Conductivity graph for test subject 35. Every Game Over (vertical red line) corresponded with a stress peak. As readable in the graph, the first game lasted a little more than one minute, the second a little less than two minutes, and the third almost three minutes.

The first graph tracked a dimension called skin conductivity of one of our test subjects. Skin conductivity provides a basic understanding of how tense or excited a test subject is by tracking the variation in moisture of his or her skin.

The results visualized in the graph above show that during the very first game, which lasted for approximately one minute and 10 seconds, Test Subject 35 was able to successfully categorize all of the beasts at the basic speed level, but failed at the first beast that would wobble faster than the initial speed. The results of the two following gameplay sessions show that excitement levels grow slightly as the difficulty level is increased, and reach extremes at Game Over states.

A game design reading of the skin conductivity graph of test subject 35 demonstrates that the ability to deal with complexity and speed progressively increases. In particular, subject 35 reached a three-minute gameplay session at her third game. This result corresponded with the aspirations of the development team and was regarded as an early success.

Overall, the test results showed that the duration and intensity of the game broadly matched the design intention of empowering the player to cope with slow beasts straight after the tutorial and to reach three-minute gameplay chunks within the first five minutes of gameplay. Nevertheless, combining the results of the biometric testing with the results of traditional questionnaires, I decided to make the game slightly slower. In the released version of the game, the rate of acceleration was also lowered and smoothed.

The second graph, by contrast, shows the variations of a second biometric parameter for the same gameplay session and in the same test subject tracked above. The second graph maps the electrical activity in the Zygomaticus Major muscle (responsible for smiling) during gameplay.


The activity of the Zygomaticus Major (smiling muscle) for test subject 35 during the same three games analyzed above. Every Game Over that was associated with stress peaks in the skin conductivity chart corresponds to a smile in this graph.

A combined reading of the graphs shows that the Game Over condition always invoked a smile in the test subject. When players were questioned about this in the post-gameplay interviews, we learned that such smiles were due to the fact that players could manipulate the cubes in an attempt to categorize the beasts until the very last pixel of the beast's tail is visible on the screen. This feature of the game produced a positive attitude that encouraged players to replay the game. The design feature gave the players a feeling of "almost having made it."

Another interesting conclusion that was inferred from the data is that beast configurations in which there was a large size disparity between heads and bodies generated more smiles. These configurations were perceived to be quirkier. The "WART-DOR" (warthog-condor) and the "RAB-PUS" (rabbit-octopus) were the configurations that elicited the most smiles in our test subjects. Although nothing was done with this information, in retrospect, we could have made these types of configurations appear more frequently in the game.


Article Start Previous Page 3 of 4 Next

Related Jobs

Firaxis Games
Firaxis Games — Sparks, Baltimore, Maryland, United States
[07.30.14]

Senior Visual Effects Artist
Nordeus
Nordeus — Belgrade, Serbia
[07.30.14]

Senior Game Designer
2K
2K — Novato, California, United States
[07.29.14]

Level Architect
Respawn Entertainment
Respawn Entertainment — San Fernando Valley, California, United States
[07.29.14]

Senior Systems Designer






Comments


Stefano Gualeni
profile image
Dear Dario, thank you for your kind comment and your interest. In case you are eager to know more, you can find an academic account of our process on our benchmark game at http://www.icemer.com (it is one of the two papers referenced at the end of the article).

As far as hardware goes, we started with this research project roughly two years ago with one of the cheapest set of biometric sensors on the market called Procompt Infiniti produced by Thought Technology: http://www.thoughttechnology.com/proinf.htm

We decided to use a very basic setup, clearly, because the original scope of our applied science and our industry partners was that of working towards the possibility of making our framework and methodologies viable for small developers. Right now, the set costs a little less than four thousand dollars, which is not too bad. :)

The biggest problem that our researchers and technicians had to solve, however, was not related to the sensors or to the creation of a neutral and isolated room to test in. The hardest problems they had to tackle consisted in bringing metrics from the game, game play videos and biometric data together in a single timeline, on a single machine where changes in psychophysiology, game performance, muscle contraction and game events could be assessed and compared.

Without a working framework capable of allowing hardware and software to communicate automatically, it is nightmarish to perform biometric analysis on video games. It was the case of Gua-Le-Ni, when our framework was just at the beginning of development. It took the technical part of our research team more than a year to develop a working and reliable version of the framework. I believe it's safe to say that, in our case, expenses and difficulties did not end with purchasing and setting up the biometric sensors.

Hopefully, commercial set of biometric sensor will soon come with a framework which is easy enough to utilize and obtain answers from. I do not know if we will be able to disclose the software we wrote and the hardware solutions we found, but I presume more technical papers will be published by the more technical people in the research team.

Once again, thanks for your interest.

David Serrano
profile image
Stefano, we know playing games will trigger a physiological responses in (most) players. But the physiological response to a game doesn't necessarily correspond to the player's opinion of the game. Or to their gameplay preferences in general, correct? Raising a player's heart rate or muscle tension could mean he or she finds the game exciting, but it also could mean the game is frustrating them or pissing them off. If I was tested while playing a game on the highest difficulty setting, I know my heart rate, blood pressure, breath rate, muscle tension, cortisol levels, etc... would spike sharply. But this wouldn't mean I found the game exciting or that I was enjoying it on any level. Because in reality, I find that playing games on higher difficulty modes sucks every second of fun, pleasure or enjoyment out of the experience. And my positive or negative opinion of games is largely determined by how well I believe the normal mode difficulty curve has been balanced for the average player the audience, and how well the casual curve was balanced for new or low experience players. So when biometric data is collected, is combining it with, or reconciling it against the player's verbal or written feedback part of the process?

Stefano Gualeni
profile image
Dear David, thank you for the very interesting question and for having shed light on the fact that I have - perhaps - not been thorough enough in the article with regard to our (multi-leyered) process. As you might imagine, writing this article for a non-specific public demanded decisions about what to omit or simplify about the ways in which we gathered, collated and interpreted data.

Your answer could be, very synthetically, found in this sentence of page 2: “To complement a wider quality assurance campaign based on questionnaires, interviews, blind-testing and hard-core performance tests, the Dutch research team at NHTV Breda University of Applied Sciences ran an initial series of biometric tests on Gua-Le-Ni. The aim of these initial tests was to structure a testing methodology incorporating the added perspective of biometry.”

I am thankful for your question because it allows me to elaborate a little more on our work.

So, David, the first thing that you need to keep in mind while reading about our tests is that each subject that tested our game was also exposed to other TWO video games in the same sector and genre of Gua-Le-Ni (casual time-base action video games). In that way we could obtain biometric data about our competitors and have a rough base to compare our game against. However, we did not tell the test subjects that one of the games was developed internally.

On top of that, we administered to every participants mini in-game questionnaires to be filled in quickly between games (normally administered upon ‘game overs’ and to be rapidly filled-in). At the end of each game session with one of the three tested games (ours plus the two control ones), a more thorough questionnaire about the general game experience – also known as a GEQ – was filled in by our guinea pigs.

At the end of the process described above, we would informally discuss with each participants the merits of the games, making notes about the difficulties, the feelings, the interfaces and generally anything they wanted to disclose about their experiences. The interviews mostly focused on Gua-Le-Ni, which (depending on the subject) was either the first, the second or the third game of the series of action-based casual games they played.

Interviews were the most useful for me as a designer, but they were also poorly reliable. As it turns out, players tend to have a very selective and distorted set of memories about their game experience. The specific literature informs us that they can remember very well the beginning and the end of the experience, and perhaps register accurately a particular event that happened during gameplay, but the rest of the playing session is usually vague in their cognition and is mostly re-constructed a posteriori. The vagueness and the cognitive blanks could be filled, in our case, with metrics, biometrics and videos. In that way, we can complement their feelings with an objective tracking of the game sessions from both an in-game performance point of view and a bodily one. Besides, interviews and records of the game states are normally crucial in determining how the bodily signals should be interpreted (or at least suggest a way in which they could be read.

The riddle of the smiles during ‘game overs’ that was cited as an example in the article was solved precisely during informal interviews, where the players specified that the end of their games were received positively. During the interviews they specified clearly that they wanted to keep going on with the game and that the gradual disappearance of the beast in play behind the curled page always left them with the feeling of having ‘almost solved it’, hence their smiles.

The positive valence of those stress spikes remained a mystery until we compared our notes about the interviews with the actual stress graphs. Interesting, isn’t it?

David Serrano
profile image
Stefano - Thank you for taking the time to reply. Yes, it's very interesting because during my career in magazine publishing, we went through a similar period where new techniques and systems were implemented with the goal of reducing or eliminating the level of subjectivity and guess work from creative and technical processes. So I see many parallels between the problems you're addressing with biometric data and the problems other industries addressed through a marriage of statistical process control and science. I think the game industry will eventually create similar systems, standards and procedures. But implementing them will be a slow, tedious process with a steep learning curve. But the end results are absolutely worth it.

Susan O'Connor
profile image
Interesting, thanks for this

Stefano Gualeni
profile image
Dear Susan, thank you for having read and commented my article.
In case any of you were interested in knowing more about our ongoing process, our framework for biometric analysis, or simply feel like meeting up, shaking hands and the like, well... You might be interested in knowing that I will be one of the speakers at the upcoming 2013 Games User Research Summit in San Francisco on March the 26th. (http://www.gur2013.org/)

In case you are planning to attend, feel free to contact me. Also, a new game might be on the way...


none
 
Comment: