Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN
arxiv(2024)
Abstract
This study focuses on Hand Gesture Recognition (HGR), which is vital for
perceptual computing across various real-world contexts. The primary challenge
in the HGR domain lies in dealing with the individual variations inherent in
human hand morphology. To tackle this challenge, we introduce an innovative HGR
framework that combines data-level fusion and an Ensemble Tuner Multi-stream
CNN architecture. This approach effectively encodes spatiotemporal gesture
information from the skeleton modality into RGB images, thereby minimizing
noise while improving semantic gesture comprehension. Our framework operates in
real-time, significantly reducing hardware requirements and computational
complexity while maintaining competitive performance on benchmark datasets such
as SHREC2017, DHG1428, FPHA, LMDHG and CNR. This improvement in HGR
demonstrates robustness and paves the way for practical, real-time applications
that leverage resource-limited devices for human-machine interaction and
ambient intelligence.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined