Routing of Public and Electric Transportation Systems Using Reinforcement Learning

Abstract

Traditional public transportation mostly uses fixed schedules to meet society’s needs reliably. Alternatively, one can design a dynamic policy to update the schedule according to the state of the network. Using such policy, any dynamic changes in demand, travel times, traffic, geometry, or emergency modes will be reflected in the real-time scheduling. In this study, the application of reinforcement learning for public transportation scheduling and routing is proposed considering electric vehicles. Reinforcement learning is a right choice for online decision-making in a time-variant setting. Reflecting electric vehicles’ characteristics on public transportation scheduling supports the necessity of considering a dynamic scheduling policy, such as vehicle-to-grid transaction capability or dynamic charging strategies. The proposed scheduling methodology is evaluated for a customary six-stop network. The trained agent takes actions based on the number of waiting people, the drop-off requests, battery level, time, and the current location. The assumed models for the traffic load effect, the passenger flow per stops, and the electricity price add nonlinearity to the problem in which this algorithm will behave more conveniently compared to a fixed schedule setting. Using this method, the electric bus can choose the next destination, the optimal driving speed for each interval, and the amount of electricity to charge or to sell for optimal financial gain when connected to the charging infrastructure. The proposed environment is built numerically, and a learning-based agent is trained for this purpose. Our study supports that the optimal dynamic agent outperforms a fixed schedule policy in different modes of network performance, particularly, when the network undergoes abrupt demand changes.