A real-time railway fastener inspection method using the lightweight depth estimation network

Fasteners are critical components of railways that maintain the rail tracks in a fixed position. Their failure can lead to serious accidents such as train derailments, so their condition needs to be inspected periodically. Conventional image-based inspection methods fail to take full advantage of the structural features of fasteners, making them less robust in real-world environments. This paper presents a new approach for real-time fastener inspection by (1) extracting fastener regions using the YOLOv3-tiny network (2) proposing and pruning a lightweight and encoder–decoder network architecture for inferring depth information from a single RGB image of fasteners (3) fusing the RGB-D features for inspection. Compared with the image-based SVM, the F1 of RGB-D fusion-based SVM increases from 94.34% to 95.83%, illustrating the improvement of additional depth information for fastener defect inspection. The inspection system runs at 11.9 FPS, which enables real-time inspection of railway fasteners.