Multi-spectrum Fusion Towards 3D Human Pose Estimation Using mmWave Radar
Proceedings of 2022 Chinese Intelligent Systems Conference(2022)
Abstract
Range-Doppler Map (RDM) and Range-Angle Map (RAM) are two kinds of representation forms of spectrum data commonly used in human pose estimation with mmWave radars. It is often that using them separately will fail to work well when the reflection signals received are insufficient for reconstructing a non-destructive human pose. To remediate the destructiveness of human pose estimations with only RDM or RAM data, this paper will explore a novel paradigm of data fusion to take full use of RDM and RAM data in human pose estimation. In doing this, two issues arise: one is how to associate the features of the two spectrums and the other is how to remove redundant features. To address these issues, inspired by the structure of Transformer encoder, a novel multi-spectrum fusion method is presented, in which a cross-attention fusion mechanism is used to facilitate the corresponding velocity to angle at the same distance region but without introducing redundant distance features. Experiment demonstrated the effectiveness of our method.
MoreTranslated text
Key words
Multi-spectrum fusion, 3D human pose estimation, Transformer, Millimeter wave radar
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined