Single-track railway scheduling with a novel gridworld model and scalable deep reinforcement learning

This study proposes a deep reinforcement learning approach for bi-direction single-track railway scheduling called DRLA-eTGM. The goal is to define the departure times and track allocation without conflict for all trains on the line given their speed, priority, origin, and destination while minimizing the total priority-weighted dwelling time. A novel time-space-capacity gridworld model (TGM), which contains a two-dimensional state representation to dynamically describe the distribution of constraints and trains in single-track railway scheduling, is proposed. Moreover, DRLA-eTGM innovatively extracts and exploits features from the TGM state automatically with deep convolutional neural networks to enhance scheduling decision strategies. Further, a shaping technique inspired by dynamic programming is adopted in DRLA-eTGM. Computational experiments are presented to verify the effectiveness of the proposed approach using hypothesized instances and a busy passenger-cargo single-track railway corridor in China. The experimental results show that DRLA-eTGM outperforms some heuristic and reinforcement learning algorithms, especially in real-world instances. Compared to some deep reinforcement learning algorithms, DRLA-eTGM shows robustness and efficiency owing to its partial scalable observation and parameter sharing mechanisms. This study provides a new approach to exploiting constraint information in railway scheduling and a prototype system for automatic single-track railway scheduling.