An Implementation of 3D Registration of Objects for AR Application Using HMD

For complex assembly, good assembly instruction makes assembling operations and trainings more effective and efficient. Instead of reading instruction manuals, it is a potential way for improving assembling operations using augmented reality technology. For example, CAD models can be displayed at the installing location to guide operators through fixed display or HMD (Head-Mounted Display). To achieve it, we first need the transformation relationship between the real and virtual world. In this research, a scheme is proposed to evaluate the transformation relationship only using HMD of Microsoft HoloLens for mixed reality with an embedded depth camera. Without using AR markers or other devices, the transformation relationship can be constructed by aligning the CAD model of a reference object to the corresponding physical object and calculating the transformation matrix. The alignment consists of two steps: rough alignment and precise alignment. Rough alignment is executed to roughly map the CAD model to the position of the real object by user’s hand manipulation, a function of image-based hand identification to control the posture of the CAD model. Afterwards, precise alignment is conducted to minimize the difference of the position and the orientation by point-to-plane ICP. Dense point cloud, digital information of the physical object, is accessed from the depth camera in HoloLens to do point-to-plane ICP with the CAD model. With this preliminary development, this scheme would be potential for not only assembly instruction but also assembly inspection to give instructions for the correction when the misassembly occurs.