Abstract
Effective police patrol scheduling is essential in projecting police presence and ensuring readiness in responding to unexpected events in urban environments. However, scheduling patrols can be a challenging task as it requires balancing between two conflicting objectives namely projecting presence (proactive patrol) and incident response (reactive patrol). This task is made even more challenging with the fact that patrol schedules do not remain static as occurrences of dynamic incidents can disrupt the existing schedules. In this paper, we propose a solution to this problem using Multi-Agent Reinforcement Learning (MARL) to address the Dynamic Bi-objective Police Patrol Dispatching and Rescheduling Problem (DPRP). Our solution utilizes an Asynchronous Proximal Policy Optimization-based (APPO) actor-critic method that learns a policy to determine a set of prescribed dispatch rules to dynamically reschedule existing patrol plans. The proposed solution not only reduces computational time required for training, but also improves the solution quality in comparison to an existing RL-based approach that relies on heuristic solver.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Canese, L., et al.: Multi-agent reinforcement learning: a review of challenges and applications. Appl. Sci. 11(11), 4948 (2021)
Chase, J., Phong, T., Long, K., Le, T., Lau, H.C.: Grand-vision: an intelligent system for optimized deployment scheduling of law enforcement agents. In: Proceedings of the International Conference on Automated Planning and Scheduling, vol. 31, pp. 459–467 (2021)
Chen, X., Tian, Y.: Learning to perform local rewriting for combinatorial optimization. In: 33rd Conference on Neural Information Processing Systems (2019)
Chen, Y., et al.: Can sophisticated dispatching strategy acquired by reinforcement learning? - a case study in dynamic courier dispatching system. In: AAMAS 2019: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (2019)
Dewinter, M., Vandeviver, C., Vander Beken, T., Witlox, F.: Analysing the police patrol routing problem: a review. ISPRS Int. J. Geo Inf. 9(3), 157 (2020)
Espeholt, L., et al.: IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures. In: Proceedings of the 35th International Conference on Machine Learning, pp. 1407–1416 (2018)
Ghiani, G., Guerriero, F., Laporte, G., Musmanno, R.: Real-time vehicle routing: solution concepts, algorithms and parallel computing strategies. Eur. J. Oper. Res. 151(1), 1–11 (2003)
Gronauer, S., Diepold, K.: Multi-agent deep reinforcement learning: a survey. Artif. Intell. Rev. 55(2), 895–943 (2022)
Joe, W., Lau, H.C., Pan, J.: Reinforcement learning approach to solve dynamic bi-objective police patrol dispatching and rescheduling problem. In: Proceedings of the International Conference on Automated Planning and Scheduling, vol. 32, pp. 453–461 (2022)
Li, W., Ni, S.: Train timetabling with the general learning environment and multi-agent deep reinforcement learning. Transp. Res. Part B: Methodol. 157, 230–251 (2022)
Liu, C.L., Chang, C.C., Tseng, C.J.: Actor-critic deep reinforcement learning for solving job shop scheduling problems. IEEE Access 8, 71752–71762 (2020)
OroojlooyJadid, A., Hajinezhad, D.: A review of cooperative multi-agent deep reinforcement learning. arXiv preprint arXiv:1908.03963 (2019)
Ray: Ray RLlib. https://www.ray.io/rllib
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M.: Deterministic policy gradient algorithms. In: Proceedings of the International Conference on Machine Learning, pp. 387–395. PMLR (2014)
Watanabe, T., Takamiya, M.: Police patrol routing on network Voronoi diagram. In: Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication, pp. 1–8 (2014)
Zhang, C., Song, W., Cao, Z., Zhang, J., Tan, P.S., Chi, X.: Learning to dispatch for job shop scheduling via deep reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1621–1632 (2020)
Zhang, K., Yang, Z., Başar, T.: Multi-agent reinforcement learning: a selective overview of theories and algorithms. In: Vamvoudakis, K.G., Wan, Y., Lewis, F.L., Cansever, D. (eds.) Handbook of Reinforcement Learning and Control. SSDC, vol. 325, pp. 321–384. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60990-0_12
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wong, S., Joe, W., Lau, H.C. (2023). Dynamic Police Patrol Scheduling with Multi-Agent Reinforcement Learning. In: Sellmann, M., Tierney, K. (eds) Learning and Intelligent Optimization. LION 2023. Lecture Notes in Computer Science, vol 14286. Springer, Cham. https://doi.org/10.1007/978-3-031-44505-7_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-44505-7_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44504-0
Online ISBN: 978-3-031-44505-7
eBook Packages: Computer ScienceComputer Science (R0)