Using Think-Aloud Protocol in Immersive VR Evaluations

There is a growing research interest concerning the use of VR as a medium in which to stage experimental evaluations. However, it is still unclear whether conventional usability evaluation methods can be directly applied to virtual reality evaluations and whether they will lead to similar insights when compared to the results of conventional usability real-world laboratory studies. Furthermore, the impact of graphic and interaction fidelity of the virtual prototype on the evaluation result is largely unexplored. Hence, we conducted two user studies with 44 participants. Results in the first study show that 61% of the reported usability problems were shared by both versions, highlighting the potential of Immersive Virtual Reality Evaluations (IVREs) as a method to evaluate early design concepts before committing to a physical prototype. In the second study, we found that usability issues related to the manual operation of the interactive device might not be identified if the interaction is implemented via supernatural techniques. In terms of graphical implications, designers should focus on those visual elements that communicate information to users. We discuss these implications and provide guidelines for deploying IVREs.