Efficient evaluation of misalignment between real and virtual objects for HMD-Based AR assembly assistance system

Augmented reality (AR) technology is a promising tool for field service applications such as assembly. AR can show computer-aided design (CAD) models of assembly parts at installation locations. This provides a means for delivering assembly instructions to an operator. Furthermore, evaluating the assembly process can be beneficial for preventing assembly errors. Field service engineers who perform assembly operations may choose to use a head-mounted display (HMD) for its good portability and hands-free operation. In this study, we propose a method for efficiently evaluating the misalignment between real and virtual objects, which is a mismatch between the two objects' poses, in an HMD-based AR assembly assistance system. By using a depth camera in the HMD, we evaluate the misalignment by comparing the depth maps of the real and virtual worlds. With our preliminary prototype, the system can evaluate misalignment with an accuracy of +/- 1 cm at a rate of 30 fps in real time. Moreover, we also propose a coordinate calibration method and establish a basic HMD-based AR assembly assistance system to demonstrate the methods. The efficient evaluation of misalignment could help to monitor operations to prevent assembly errors and decrease operation time.