Towards Recognizing Person-Object Interactions Using A Single Wrist Wearable Device

Activity recognition (AR) is an important part of context aware applications. In this paper, we focus on an indirect AR method: by sensing the objects that a person is using. Objects that provide functional utility to their user, also indicate the type of activity that their user is doing. For example, the use of a hair dryer indicates that its user is grooming their hair. In this paper, we discuss an approach to sense the objects that the person interacts with, using only a single wearable device on the wrist of the person. Wearable devices typically have an IMU sensor, which can sense several aspects of the person's hand gestures, such as acceleration, and orientation. We collect a dataset of 17 different object interaction gestures using 5 participants in a test home. We evaluate the object gestures using supervised and unsupervised machine learning approaches. Our study reveals that we can recognize object interactions with 83-91% accuracy in the supervised approach, and 58-66% accuracy in the unsupervised approach.