Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Gamasutra: The Art & Business of Making Gamesspacer
Successful Playtesting In Swords & Soldiers
View All     RSS
October 26, 2020
arrowPress Releases
October 26, 2020
Games Press
View All     RSS







If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

Successful Playtesting In Swords & Soldiers


July 28, 2010 Article Start Previous Page 3 of 3
 

Results, and What Ronimo Did With Them

After the test, Ronimo implemented the findings into the final design according to the feedback we provided. Thankfully, there weren't many issues that needed fixing.

In-game controls worked well. The primary actions of casting spells and building soldiers worked like a breeze. Possibly, using two radial menus had a side benefit: it immediately clarified the distinction between spells and soldiers.

The most important issue was found when in the menu for upgrading soldiers and spells: the tooltip explaining a specific upgrade was displayed on top of the other icons, making it hard to quickly scan for new upgrades, pick one, and continue gaming. Ronimo elegantly solved this by letting the tooltip fade after half a second or so.

Players were caused some confusion by the game's depiction of gold as a golden circle marked with an X. Some players mistook that icon for the X button on their controller, and were trying to mine the gold by pressing the X button. This was fixed by changing the symbol into a plus.

We saw quite a few players barely read the tutorial instructions. Ronimo shortened that text, to make sure it was more to the point.


Eye tracking: this was a player in his tenth minute of playing. First a short fight, then building soldiers and upgrading. To see a video of this session, click here.

Menu Structure Lacks Feed-Forward. Overall, players easily navigated through the menu. Some players, however, overlooked options like changing the difficulty level or changing tribes.

This was possibly due to lack of feed-forward: the game menus didn't make clear what would happen when you clicked on an item. Since restructuring the menu wasn't feasible, Ronimo let the cursor start at the menu-item which was overlooked during the test. The idea is to focus attention on the button.

What We Learned

Eight players is more than enough. There was little extra benefit from the last two players. Six, maybe five, people would have sufficed to answer the most important questions. However, it's risky to generalize this "six is enough for a playtest rule", since the scope of this test was quite narrow (no focus on gameplay, multiplayer, or story) and games vary widely in size and complexity.

Nevertheless, based on this and some previous playtests, we feel that for a medium-sized game, usability issues concerning controller, menus, and HUD can be identified by observing about six players.

Controller cam only useful at post-test interview. Only during the interview afterwards, when participants explained their experience with the controls, did we find the controller camera to be of use. During play, it was almost impossible to keep track of which button is pressed at which specific moment while also focusing on the player cam and the gameplay. Also, the low framerate of the controller cam and the slight delay rendered it almost useless.

It would be good to have some form of real-time button logging, to display the buttons which are pressed in a timeline. Perhaps this can be done by building it into the game itself.

Player experience: observation is good, but crude. Observing body posture and facial expression to gauge player experience was useful but limited: you can generally tell if someone has a good or a bad time, but not exactly how good or bad, or what he feels. Of course it's possible we misread some clues, so we're looking for ways to improve our observational skills.

What we did see was that posture can predict boredom or concentration: When concentrated, a player leans forward, as he is literally into the game. A bored player is slouched. Facial expression is harder to interpret as it usually comes in two flavors: either a catatonic stare or a deep frown. The stare can signify boredom or concentration; frowning can mean concentration or frustration. An excited player is easily recognized: a player yelps, grins, and clenches his fist when he discovers the destructive power of The Hammer of Thor, one of the game's magic spells.

Next time we will probably monitor the HRV and not use the controller cam. This'll give us some more insights into the player's state. Afterwards during the post-test interview, we'll change the HRV-image for the controller cam.

Conclusion

Based on the findings, it seems the developers did an excellent job designing the button mapping for the remake of Swords & Soldiers. However, some adjustments had to be made in the HUD and menu to make the game more usable.

The playtest taught us that with six players, you can probably spot most game usability issues, but for broader gameplay testing you might need more time and players. To monitor controller use, a form of button logging would be more useful than a camera focusing on the controller. An eye tracker was found to be very useful when testing menu and HUD usability. Observing the player to gauge his experience is useful, but limited. Perhaps with some training and simple biometrics, we can get a better real-time insight in what the player is experiencing.


Article Start Previous Page 3 of 3

Related Jobs

Square Enix Co., Ltd.
Square Enix Co., Ltd. — Tokyo, Japan
[10.25.20]

Experienced Game Developer
Chromatic Games
Chromatic Games — Gainesville, Florida, United States
[10.23.20]

Community Manager
Sucker Punch Productions
Sucker Punch Productions — Bellevue, Washington, United States
[10.22.20]

Producer
Airship Syndicate
Airship Syndicate — Austin, Texas, United States
[10.22.20]

Producer





Loading Comments

loader image