4. Rock the Survey
You'll almost always want to ask your players some questions after they're done playing. Try to predict these questions ahead of time and put them in survey form so that you collect the same set of data for every playtester. If you have any spontaneous follow-up questions, you can ask them when they're done taking the survey.
A carefully worded, highly focused survey can make sharing your playtest results with the team much easier. Focus on questions that directly address the goals of your playtest. Leading questions like "Was the tutorial confusing?" are much less helpful than questions that test a player's knowledge, like "Please describe what the green button does in this game."
If your playtesters are newcomers to your game or genre, they are probably unfamiliar with many of the terms and conventions that your team may take for granted -- plan accordingly by using plain, descriptive terms in your questions whenever possible.
Our surveys always include some basic player profile questions like "What games have you been playing in the last month?" to give us context. In addition, every multiple-choice question includes an optional space for written explanation.
5. Analyze in Aggregate?
After a playtest, you're going to have survey results and notes from many different playtesters and observers. Compile that data quickly and share it with your team in aggregate, without offering analysis or drawing conclusions.
Analyze the results as a group, and start on the aggregate data -- your game designer may have seen one player who thought the game was too easy, but is that what everyone else saw? If all the other players said the game was difficult then you know that "too easy" is not a trend. If you like you can still return to that playtester's feedback afterward and address it as a special case. Analyzing your data in aggregate first will guarantee that your entire team benefits from the full playtest.
Once you've got your data compiled, don't just forget about it! Identify the issues that your playtest has brought to light and prioritize your next steps. The data you collect during your playtest -- which can include anything from the player's emotional state, to the number of failed attempts to click a button -- should always help you draw conclusions and come away with action items for your team. If your data is inconclusive, consider revisiting the structure of your playtest and survey for next time.
These are the best practices we've developed at Arkadiun's New York headquarters, and they have helped us implement recurring playtesting in a consistent and reliable way. Of course, we're always improving our process, and what works well for us might not suit the needs of your studio. If you find any of these tips helpful, or have a different way of doing things, let us know in the comments section.