The Onboarding Phase in a Game for Text Labelling: Comparing the Effect of Animated vs. Textual Onboarding on Player Experience and Accuracy.

Engaging new players while collecting quality data in Games-with-a-Purpose (GWAPs) can be challenging. It is therefore crucial to understand how to design the first minutes of play to attract and engage players without compromising data quality. In our study (n = 91), we explored the impact of presenting an onboarding phase in a game for language annotation on both data accuracy and player experience. We focused on evaluating two different modalities of the onboarding phase, animated (A) and text (B), and a control (C) version where an onboarding phase was not presented. Those who were presented with an onboarding phase were briefly introduced to the game’s story and then a tutorial explaining the gameplay mechanics. During the gameplay, players’ inputs were recorded to measure the accuracy of their labels. After playing the game, players completed the Game Experience Questionnaire to evaluate their player experience. The results indicate that while the accuracy of the data collected is not significantly different amongst the three groups, players experienced significantly higher frustration in the text version of the onboarding phase than in the control. This paper suggests that GWAP designers should examine whether investing in an onboarding phase can be justified for their games, or whether the game mechanics can be simply explored by players.