An adaptable system for RGB-D based human body detection and pose estimation

HighlightsDoes not require pre-processing by background subtraction and no initialization poses.Online learned appearance model combining color with depth-based labeling.Works in clutter and with body part occlusions because of underlying kinematic model.RDF training, data generation and cluster-based learning, that enables retraining. Human body detection and pose estimation is useful for a wide variety of applications and environments. Therefore a human body detection and pose estimation system must be adaptable and customizable. This paper presents such a system that extracts skeletons from RGB-D sensor data. The system adapts on-line to difficult unstructured scenes taken from a moving camera (since it does not require background subtraction) and benefits from using both color and depth data. It is customizable by virtue of requiring less training data, having a clearly described training method, and a customizable human kinematic model. Results show successful application to data from a moving camera in cluttered indoor environments. This system is open-source, encouraging reuse, comparison, and future research.