Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Gamasutra: The Art & Business of Making Gamesspacer
Successful Playtesting In Swords & Soldiers
View All     RSS
October 26, 2020
arrowPress Releases
October 26, 2020
Games Press
View All     RSS

If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Successful Playtesting In Swords & Soldiers

July 28, 2010 Article Start Previous Page 2 of 3 Next

Game Lab: Observing Natural Gaming Behavior

Play sessions were held in our game lab, which looks like a living room, in order to make the player feel at home -- or at least more at ease and less self-conscious than they would be in a sterile lab setting, or playing at the developer's studio.

When a gamer can play undisturbed, without people looking over his shoulder and taking notes, he's more likely to relax and play the game as he normally would.

The lab is fitted with the following observational tools: an inconspicuous camera which focused on the player and his face to monitor player experience.

Another lightweight camera was attached to the controller to keep track of the buttons which are pressed. The eye tracker, a Tobii T120, looks like a regular 17-inch screen but has an infrared camera following eye movements.

It displays the gameplay and the player's gaze. Game audio and spontaneous player comments were also captured.

From a separate observation room, Ronimo and Valsplat observed gameplay and behavior, using a list of research questions.

The observation room, with gameplay and player gaze projected on the left, controller and user cam on the right.

Overview of a Play Session

Each session took about 45 minutes. All eight sessions were held consecutively, with breaks between sessions to recap with Ronimo and tweak the test set-up.

A session started with the participant signing an NDA; then I configured the eye tracker and explained the test. To make the player feel relaxed, I explained that it's not the player, but instead the game which is tested, and we need the player's help to improve the game.

Also, I emphasized my independence from the developer: I won't be offended if they tell me the game sucks. The only instruction I gave was to play the game as they normally would. Thinking aloud was not encouraged, since it may interfere with normal gaming. Then I joined my colleague and two Ronimo developers in the observation room.

When we felt we learned everything from the participant, usually after 30 minutes, I re-entered the lab for an evaluation interview. In this semi-structured interview, participants were asked about the game, how the controls felt, and if they could work with the interface. Also, participants were given a few tasks, such as changing game difficulty or setting up a multiplayer game.

Then, the player received his incentive and could go home, and the lab was prepared for the next session.

Gauging the Player Experience: Part Art, Part Science, Mainly Minefield

Reliably measuring player experience is hard. For a solid scientific measurement, biometrics can be valuable, but this requires at least 40 players and (expensive) tools to measure psychophysical data like brain activity, sweat, or heart rate variability. You also need strict conditions and must invest quite some time in data analysis afterwards. This isn't feasible for this kind of playtest -- but we wanted some kind of real-time, quick and dirty way to see what a player experiences.

With a previous playtest we had an ear clip measuring heart rate variability (HRV), which can indicate emotional state. For example, an HRV spike can signify stress or happiness. The ear clip was non-intrusive but not always accurate (shaking your head distorts the data). Nevertheless, the tool gave us some easy to interpret real-time charts which needed to be interpret in context of the game. For this test however, we didn't measure HRV: we figured that yet another display would be too much to keep track of.

Instead, we gauged the player experience by watching a participant's body language and reading his or her facial expression, in context of the game. Simply put, we watched if players were bored, concentrated, excited or frustrated. But you can't rely on observation alone. Some people are very expressive when playing; others are more reserved. So after the test, we asked players how they felt while playing. However, this has disadvantages; it can be hard to accurately remember emotions you had half an hour ago, or verbalize them.

Eye Tracking: See What They See

The eye tracker is a useful tool when testing menu usability and in-game HUD. Even though no statistical conclusions can be drawn from the eye tracking data of eight participants, observing how a player scans the screen, what they see and miss, gives a better insight as to why someone completely misreads a menu or overlooks a button. For observing gameplay, however, it was less useful: it's hard to follow a user's eyes in a heated clash.

It was interesting to see how some guys automatically looked at the Viking women.

Article Start Previous Page 2 of 3 Next

Related Jobs

Square Enix Co., Ltd.
Square Enix Co., Ltd. — Tokyo, Japan

Experienced Game Developer
Chromatic Games
Chromatic Games — Gainesville, Florida, United States

Community Manager
Sucker Punch Productions
Sucker Punch Productions — Bellevue, Washington, United States

Airship Syndicate
Airship Syndicate — Austin, Texas, United States


Loading Comments

loader image