K-glass: Real-time markerless augmented reality smart glasses platform

Recently, augmented reality (AR) has been actively studied with the proliferation of advanced displays such as a head mounted display (HMD). Nevertheless, it was almost impossible for the conventional HMDs to process the complex AR algorithm in real time, because it requires highly accurate registration and positioning of virtual objects in the real environment. In this paper, we present the world's first markerless AR HMD system equipped with a high-throughput AR processor. Thanks to the dedicated AR processor, it can recognize general objects without any help of markers and give 3D information to the user in real time (30fps). Also, the processor adopts dynamic voltage and frequency scaling as a power managing scheme for low power consumption. As a result, it sustains one-day long real-time AR operation consuming 381mW on average when it performs the full chain of AR processing including object recognition, camera pose estimation, and 3D graphics rendering on a HD (720p) input video stream.