Application of Vision-Based Artificial Intelligence in Creating a Contactless Interaction with Immersive Environments

Virtual reality (VR) is one of the building information modeling (BIM) use cases that can benefit a project from the early stages of design to the construction and operation phases. VR enables designers, contractors, and owners to experience interacting with a facility before it is constructed. Currently, interaction with VR environments takes place through controllers. The equipment helps users to navigate virtual facilities and interact with the surrounding objects. Although controllers address the need for creating an interactive virtual environment, they make the experience less realistic, since real-life interactions take place through body movements, not by/through pressing buttons. This paper reports a project in which vision-based artificial intelligence (AI) technology is used to create a contactless connection between users and virtual environments. In this project, a series of body movements and gestures are mapped to the input commands of a BIM-based VR software. The paper overviews the state of the art in remote human-computer interaction (HCI) technologies and discusses both the potential and challenges of applying HCI technologies to immersive VR environments. It then introduces the architecture of the developed, contactless virtual environment and reports a case study of using this technology in an immersive setting.