Histogram of 3D Facets

AbstractA new 3D local shape descriptor for hand gesture and human action recognition from depth sensors.Extensive results demonstrate the outstanding discriminative power of the proposed descriptor.The proposed descriptor is compact in dimension, which is thus economic to compute.A Dynamic Programming based Temporal Segmentation framework to represent dynamic depth videos. The recent successful commercialization of depth sensors has made it possible to effectively capture depth images in real time, and thus creates a new modality for many computer vision tasks including hand gesture recognition and activity analysis. Most existing depth descriptors simply encode depth information as intensities while ignoring the richer 3D shape information. In this paper, we propose a novel and effective descriptor, the Histogram of 3D Facets (H3DF), to explicitly encode the 3D shape information from depth maps. A 3D Facet associated with a 3D cloud point characterizes the 3D local support surface. By robust coding and circular pooling 3D Facets from a depth map, the proposed H3DF descriptor can effectively represent both 3D shapes and structures of various depth maps. To address the recognition problems of dynamic actions and gestures, we further extend the proposed H3DF by combining it with an N-gram model and dynamic programming. The proposed descriptor is extensively evaluated on two public 3D static hand gesture datasets, one dynamic hand gesture dataset, and one popular 3D action recognition dataset. The recognition results outperform or are comparable with state-of-the-art performances.