Keypoint-based contextual representations for hand pose estimation

Multimedia Tools and Applications(2024)

Cited 0|Views6
No score
Abstract
Most current methods for the hand pose estimation ignore the pixel-level relationship of hand keypoints, e.g. four specific keypoints in the same finger can form a semantically continuous area at pixel level. To make full use of pixel-level semantic information extracted from the origin RGB image, we propose a novel keypoint-based contextual representation(KCR) scheme for hand pose estimation, which can leverage pixel-level continuous contextual features based on the hand structure without using any additional labeling information. To extract hand structure information from the contextual features, we creatively design a novel keypoint representation and finger representation scheme by fusing the keypoints feature in a specific group. Then, the cross-attention mechanism is used to calculate the relation between the finger representations and contextual features to improve the feature integration. The augmented feature contains more hand structure information for the final hand pose estimation. Experimental results demonstrate that our method achieves competitive performance on various 2D and 3D hand pose estimation benchmarks.
More
Translated text
Key words
Hand pose estimation,Gesture recognition,Relational context,Keypoint heatmap,Object Detection
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined