IDCam: Precise Item Identification for AR Enhanced Object Interactions

Augmented reality (AR) promises to revolutionize the way people interact with their surroundings by seamlessly overlaying virtual information onto the physical world. To improve the quality of such information, AR systems need to identify the object with which the user is interacting. AR systems today heavily rely on computer vision for object identification; however, state-of-the-art computer vision systems can only identify the general object categories, rather than their precise identity. In this work, we propose IDCam, a system that fuses RFID and computer vision for precise item identification in AR object-oriented interactions. IDCam simultaneously tracks users’ hands using a depth camera and generates motion traces for RFID-tagged objects. The system then correlates traces from vision and RFID to match item identities with user interactions. We tested our system through a simulated retail scenario where 5 participants interacted with a clothing rack simultaneously. In our evaluation study deployed in a lab environment, IDCam identified item interactions with an accuracy of 82.0% within 2 seconds.