Understanding Pitfalls and Opportunities of Applying Heuristic Evaluation Methods to VR Training Systems: An Empirical Study

The usability of virtual reality (VR) training applications is crucial for their success, but examining the usability in the early development stages remains challenging. A realistic and plausible solution would be revisiting and reconciling Heuristics Evaluation (HE) methods among the most widely used usability inspection methods in the human-computer interaction (HCI) domain. While research on studying and using HE methods is growing within the VR domain, few studies have considered the novel VR environment challenges new requirements for fitting HE methods to the context and applying them effectively. To this end, we conducted a user study with 14 evaluators using the standard HE methods to complete two HE sessions for a VR training application. We identified five critical challenges that evaluators encountered in the HE process by observing and interviewing them. Based on our findings, we discuss the importance of considering an easy-to-use heuristic set, how we can facilitate the HE procedures in the VR context, and the opportunities for developing HE-supporting tools.