3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera
CoRR(2023)
摘要
3D hand tracking from a monocular video is a very challenging problem due to
hand interactions, occlusions, left-right hand ambiguity, and fast motion. Most
existing methods rely on RGB inputs, which have severe limitations under
low-light conditions and suffer from motion blur. In contrast, event cameras
capture local brightness changes instead of full image frames and do not suffer
from the described effects. Unfortunately, existing image-based techniques
cannot be directly applied to events due to significant differences in the data
modalities. In response to these challenges, this paper introduces the first
framework for 3D tracking of two fast-moving and interacting hands from a
single monocular event camera. Our approach tackles the left-right hand
ambiguity with a novel semi-supervised feature-wise attention mechanism and
integrates an intersection loss to fix hand collisions. To facilitate advances
in this research domain, we release a new synthetic large-scale dataset of two
interacting hands, Ev2Hands-S, and a new real benchmark with real event streams
and ground-truth 3D annotations, Ev2Hands-R. Our approach outperforms existing
methods in terms of the 3D reconstruction accuracy and generalises to real data
under severe light conditions.
更多查看译文
关键词
hand pose estimation,event-based vision
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要