Glass Detection in Simultaneous Localization and Mapping of Mobile Robot Based on RGB-D Camera

Simultaneous Localization and Mapping (SLAM) is a critical technology for modern mobile robotics. Robots with sensors like monocular, binocular, RGB-D cameras, and LIDAR can build maps of unknown indoor environments through SLAM and locate themselves in those maps. However, the presence of glass surfaces, which is a common feature in indoor scenes, can pose a challenge to the accuracy of SLAM due to their high transparency and reflectivity, which make it difficult for sensors to detect them. We propose an approach for detecting glass panels using RGB-D sensors and integrating them into maps with high precision using ORBSLAM2 and semantic segmentation. By leveraging RGB-D cameras to obtain RGB images and depth maps, and utilizing ORBSLAM2 to acquire map points and camera pose information, our approach locates glass panels accurately. The pixel-level classification of semantic segmentation is obtained based on the RGB image, and the position of the large plane is obtained based on the depth image. The above results are fused to obtain a more complete map containing the glass surface. The resulting maps include glass surfaces calculated within ±3° of angle error and ±0.1m of distance error. The results verify the effectiveness of the system and provide a solid basis for future research in this area.