A Deep Learning Approach for Automated Detection of Railroad Trespassers

Trespassing along the railroad rights-of-way is the leading cause of rail-related deaths, with more than 400 annual fatalities in the US. Detection of such events using video camera systems is critical for railroad safety improvements while challenging due to the immense labor costs required for the processing and monitoring of a large number of streamed video files. The objective of this study is to develop a deep learning based automatic trespassing detection process using streamed railway surveillance video data, which will assist in identifying, characterizing, and alerting potential trespassers in the region of interest (ROI). In this research, the You Only Look Once (YOLO) v4 and Deep SORT are implemented for trespassing detection and tracking. The result shows that all trespassers in the video were detected without false negative or false positive. The detection results can be applied to the trespasser detection and alerting systems for sending a timely alert to the trespasser, sending the trespassing information to railroad safety officials, and saving trespassing event video data to the database. It is anticipated that this work could assist railroad agencies to improve railroad security based on real-time detection results.