Abstract
Urban communities rely heavily on the system of interconnected critical infrastructures. The interdependencies in these complex systems give rise to vulnerabilities that must be considered in disaster mitigation planning. Only then will it be possible to address and mitigate major critical infrastructure disruptions in a timely manner.
This paper describes an intelligent decision making system that optimizes the allocation of resources following an infrastructure disruption. The novelty of the approach arises from the application of Monte Carlo estimation for policy evaluation in reinforcement learning to draw on experiential knowledge gained from a massive number of simulations. This method enables a learning agent to explore and exploit the available trajectories, which lead to an optimum goal in a reasonable amount of time. The specific goal of the case study described in this paper is to maximize the number of patients discharged from two hospitals in the aftermath of an infrastructure disruption by intelligently utilizing the available resources. The results demonstrate that a learning agent, through interactions with an environment of simulated catastrophic scenarios, is capable of making informed decisions in a timely manner.
Chapter PDF
Similar content being viewed by others
References
C. Arboleda, D. Abraham and R. Lubitz, Simulation as a tool to assess the vulnerability of the operation of a health care facility, Journal of Performance of Constructed Facilities, vol. 21(4), pp. 302–312, 2007.
C. Arboleda, D. Abraham, J. Richard and R. Lubitz, Impact of interdependencies between infrastructure systems in the operation of health care facilities during disaster events, Proceedings of the Twenty-Third Joint International Conference on Computing and Decision Making in Civil and Building Engineering, pp. 3020–3029, 2006.
C. Arboleda, D. Abraham, J. Richard and R. Lubitz, Vulnerability assessment of health care facilities during disaster events, Journal of Infrastructure Systems, vol. 15(3), pp. 149–161, 2009.
G. Atanasiu and F. Leon, Agent-based risk assessment and mitigation for urban public infrastructure, Proceedings of the Sixth Congress on Forensic Engineering, pp. 418–427, 2013.
E. Bonabeau, Agent-based modeling: Methods and techniques for simulating human systems, Proceedings of the National Academy of Sciences, vol. 99(3), pp. 7280–7287, 2002.
F. Daniel, India power cut hits millions, among world’s worst outages, Reuters, July 31, 2012.
M. Khouj and J. Marti, Modeling Critical Infrastructure Interdependencies in Support of the Security Operations for the Vancouver 2010 Olympics, Technical Report, Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada, 2010.
M. Khouj, S. Sarkaria and J. Marti, Decision assistance agent in real-time simulation, International Journal of Critical Infrastructures, vol. 10(2), pp. 151–173, 2014.
K. Kowalski-Trakofler, C. Vaught and T. Scharf, Judgment and decision making under stress: An overview for emergency managers, International Journal of Emergency Management, vol. 1(3), pp. 278–289, 2003.
J. Marti, J. Hollman, C. Ventura and J. Jatskevich, Dynamic recovery of critical infrastructures: Real-time temporal coordination, International Journal of Critical Infrastructures, vol. 4(1/2), pp. 17–31, 2008.
J. Marti, C. Ventura, J. Hollman, K. Srivastava and H. Juarez-Garcia, i2Sim modeling and simulation framework for scenario development, training and real-time decision support of multiple interdependent critical infrastructures during large emergencies, presented at the NATO RTO Symposium on How is Modeling and Simulation Meeting the Defense Challenges out to 2015, 2008.
J. Marti, E. Yanful and M. Ulieru, Disaster Response Network Enabled Platform, CANARIE Project Final Report, Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada, 2012.
H. Min, W. Beyeler, T. Brown, Y. Son and A. Jones, Toward modeling and simulation of critical infrastructure interdependencies, IIE Transactions, vol. 39(1), pp. 57–71, 2007.
G. O’Reilly, H. Uzunalioglu, S. Conrard and W. Beyeler, Inter-infrastructure simulations across telecom, power and emergency services, Proceedings of the Fifth International Workshop on the Design of Reliable Communication Networks, 2005.
S. Rinaldi, Modeling and simulating critical infrastructure and their interdependencies, Proceedings of the Thirty-Seventh Annual Hawaii International Conference on System Sciences, 2004.
Z. Su, J. Jiang, C. Liang and G. Zhang, Path selection in disaster response management based on Q-learning, International Journal of Automation and Computing, vol. 8(1), pp. 100–106, 2011.
R. Sutton and A. Barto, Reinforcement Learning: An Introduction, Bradford/MIT Press, Cambridge, Massachusetts, 1998.
D. Thapa, I. Jung and G. Wang, Agent based decision support system using reinforcement learning under emergency circumstances, Proceedings of the First International Conference on Natural Computation, pp. 888–892, 2005.
M. Wiering and M. Dorigo, Learning to control forest fires, Proceedings of the Twelfth International Symposium on Computer Science for Environmental Protection, pp. 378–388, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Khouj, M.T., Sarkaria, S., Lopez, C., Marti, J. (2014). Reinforcement Learning Using Monte Carlo Policy Estimation for Disaster Mitigation. In: Butts, J., Shenoi, S. (eds) Critical Infrastructure Protection VIII. ICCIP 2014. IFIP Advances in Information and Communication Technology, vol 441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45355-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-662-45355-1_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-45354-4
Online ISBN: 978-3-662-45355-1
eBook Packages: Computer ScienceComputer Science (R0)