SmokePose: End-to-End Smoke Keypoint Detection

Smoke detection has been a research focus due to its application value in fire and toxic gas leakage alarms. Here we formulate a novel research paradigm for in-depth smoke analysis: modeling the smoke plume in an image with two semantic keypoints, i.e., the start-point (where the smoke comes from) and the end-point (where the smoke spreads), and localizing the keypoints through a heatmap-based detection method. A specialized dataset is developed for smoke keypoint detection, collecting images online and manually annotating the keypoints. Based on the dataset, we propose a Transformer-based model called SmokePose that employs a hierarchical Transformer encoder and a pure Transformer decoder to detect smoke keypoints in an end-to-end manner. We demonstrated the performance of the proposed SmokePose with comparative experiments and ablation studies. A further discussion on the visualization of the attention maps helps to understand the mechanism of SmokePose and to reveal essential image clues for smoke keypoint detection.