Adaptive Scanning for Improved Stacked Object Detection with RGB and LiDAR

The increasing requirements of robots in construction have brought great challenges related to understanding the complex environment and completing the downstream tasks with timelines. Focusing on stacked object detection, which is a generally complicated scenario in construction sites, this paper proposes a novel framework by using both RGB and LiDAR for object clustering with low storage occupation and high detection speed that support real-time implementation. An RGB camera is first used to capture an image of the overall scene, and a pre-trained CNN network is applied to give the rough prediction of region-of-interests (ROIs) along with their confidence scores. The ROIs are linearly sorted based on their scores to select the potential stacked areas with low confidence. The center locations of ROIs are then transferred into the LiDAR system with the calibration matrix, and a Velodyne-16 scanner is used to perform adaptive scanning on the ROIs for detailed object clustering and detection. The result shows that given the pre-detected ROIs from RGB, the scanning time and computational time of clustering could be largely reduced. Furthermore, a confidence-based criterion is illustrated to linearly determine the required scanning frames to get desired detection results.