Markov decision process model for patient admission decision at an emergency department under a surge demand

  • Hyun-Rok Lee
  • Taesik Lee


We study an admission control problem for patients arriving at an emergency department in the aftermath of a mass casualty incident. A finite horizon Markov decision process (MDP) model is formulated to determine patient admission decisions. In particular, our model considers the time-dependent arrival of patients and time-dependent reward function. We also consider a policy restriction that immediate-patients should be admitted as long as there is available beds. The MDP model has a continuous state space, and we solve the model by using a state discretization technique and obtain numerical solutions. Structural properties of an optimal policy are reviewed, and the structures observed in the numerical solutions are explained accordingly. Experimental results with virtual patient arrival scenarios demonstrates the performance and advantage of optimal policies obtained from the MDP model.


Disaster response Emergency department Admission control Mass casualty incident Markov decision process Optimal policy 



This research was supported by a Grant ‘research and development of modeling and simulating the rescues, the transfer, and the treatment of disaster victims’ (nema-md-2013-36) from the Man-made Disaster Prevention Research Center, National Emergency Management Agency of Korea.


  1. Allon G, Deo S, Lin W (2013) The impact of size and occupancy of hospital on the extent of ambulance diversion: theory and evidence. Oper Res 61(3):544–562CrossRefzbMATHGoogle Scholar
  2. Argon NT, Ziya S, Righter R (2008) Scheduling impatient jobs in a clearing system with insights on patient triage in mass casualty incidents. Probab Eng Inf Sci 22(3):301–332MathSciNetCrossRefzbMATHGoogle Scholar
  3. Arnold JL, Tsai M-C, Halpern P, Smithline H, Stok E, Ersoy G (2003) Mass-casualty, terrorist bombings: epidemiological outcomes, resource utilization, and time course of emergency needs (part I). Preshospital Disaster Med 18(3):220–234CrossRefGoogle Scholar
  4. Brumelle S, Walczak D (2003) Dynamic airline revenue management with multiple semi-Markov demand. Oper Res 51(1):137–148MathSciNetCrossRefzbMATHGoogle Scholar
  5. Christ M, Grossmann F, Winter D, Bingisser R, Platz E (2010) Modern triage in the emergency. Deutsch Ärztebl Int 107(50):892–898Google Scholar
  6. Cohen I, Mandelbaum A, Zychlinski N (2014) Minimizing mortality in a mass casualty event: fluid networks in support of modeling and staffing. IIE Trans 46(7):728–741CrossRefGoogle Scholar
  7. Deo S, Gurvich I (2011) Centralized vs. decentralized ambulance diversion: a network perspective. Manag Sci 57(7):1300–1319CrossRefzbMATHGoogle Scholar
  8. der Heide EA (2006) The importance of evidence-based disaster planning. Ann Emerg Med 47(1):34–49CrossRefGoogle Scholar
  9. Einav S, Aharonson-Daniel L, Weissman C, Freund HR, Peleg K, Israel Trauma Group (2006) In-hospital resource utilization during multiple casualty incidents. Ann Surg 243(4):533–540CrossRefGoogle Scholar
  10. Frykberg ER (2002) Medical management of disasters and mass casualties from terrorist bombings: how can we cope? J Trauma Acute Care Surg 53(2):201–212CrossRefGoogle Scholar
  11. Hick JL, Hanfling D, Cantrill SV (2012) Allocating scarce resources in disasters: emergency department principles. Ann Emerg Med 59(3):177–187CrossRefGoogle Scholar
  12. Hogan DE, Waeckerle JF, Dire DJ, Lillibridge SR (1999) Emergency department impact of the Oklahoma city terrorist bombing. Ann Emerg Med 34(2):160–167CrossRefGoogle Scholar
  13. Institute of Medicine of the National Academies (2005) Hospital-based emergency care: at the breaking point. The National Academies Press, WashingtonGoogle Scholar
  14. Jacobson EU, Argon NT, Ziya S (2012) Priority assignment in emergency response. Operat Res 60(4):813–832MathSciNetCrossRefzbMATHGoogle Scholar
  15. Jenkins JL, McCarthy ML, Sauer LM, Green GB, Stuart S, Thomas TL, Hsu EB (2008) Mass-casualty triage: time for an evidence-based approach. Prehospital Disaster Med 23(1):3–8CrossRefGoogle Scholar
  16. Kilic A, Dincer MC, Gokce MA (2013) Determining optimal treatment rate after a disaster. J Oper Res Soc 65(7):1053–1067CrossRefGoogle Scholar
  17. Korea National Emergency Medical Center (2007) Development of the guidelines for inter-hospital transfer. Ewha Womans Univ., Korea National Emergency Medical Center, SeoulGoogle Scholar
  18. Li D, Glazebrook KD (2010) An approximate dynamic programming approach to the development of heuristics for the scheduling of impatient jobs in a clearing system. Nav Res Logist 57(3):225–236MathSciNetzbMATHGoogle Scholar
  19. Mills AF, Argon NT, Ziya S (2013) Resource-based patient prioritization in mass-casualty incidents. Manuf Serv Oper Manag 15(3):361–377CrossRefGoogle Scholar
  20. Niska RW, Burt CW (2005) Bioterrorism and mass preparedness in hospitals: United States, 2003. Adv Data 364:1–14Google Scholar
  21. Peleg K, Kellerman AL (2009) Enhancing hospital surge capacity for mass casualty events. JAMA 302(5):565–567CrossRefGoogle Scholar
  22. Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming, vol 672. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  23. Ramirez-Nafarrate A, Hafizoglu AB, Gel ES, Fowler JW (2014) Optimal control policies for ambulance diversion. Eur J Oper Res 236(1):298–312MathSciNetCrossRefzbMATHGoogle Scholar
  24. Reilly MJ, Markenson D (2010) Hospital referral patterns: how emergency medical care is accessed in a disaster. Disaster Med Public Health Preparedness 4(3):226–231CrossRefGoogle Scholar
  25. Sacco WJ, Navin DM, Fiedler KE, Waddell RK II, Long WB, Buckman RF (2005) Precise formulation and evidence-based application of resource-constrained triage. Acad Emerg Med 12(8):759–770CrossRefGoogle Scholar
  26. Sacco WJ, Navin DM, Waddell RKII, Fiedler KE, Long WB, Buckman RF (2007) A new resource-constrained triage method applied to victims of penetrating injury. J Trauma Acute Care Surg 63(2):316–325CrossRefGoogle Scholar
  27. Stratton SJ, Tyler RD (2006) Characteristics of medical surge capacity demand for sudden-impact disasters. Acad Emerg Med 13(11):1193–1197CrossRefGoogle Scholar
  28. Sung I, Lee T (2016) Optimal allocation of emergency medical resources in a mass casualty incident: patient prioritization by column generation. Eur J Oper Res 252(2):623–634MathSciNetCrossRefzbMATHGoogle Scholar
  29. Timbie JW, Ringel JS, Fox DS, Pillemer F, Waxman DA, Moore M, Hansen CY, Knebel AR, Ricciardi R, Kellermann AL (2013) Systematic review of strategies to manage and allocate scarce resources during mass casualty events. Ann Emerg Med 61(6):677–689CrossRefGoogle Scholar
  30. Waeckerle JF (1991) Disaster planning and response. N Engl J Med 324(12):815–821CrossRefGoogle Scholar
  31. Xu K, Chan CW (2016) Using future information to reduce waiting times in the emergency department via diversion. Manuf Serv Oper Manag 18(3):314–331CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Industrial and Systems EngineeringKAISTDaejeonRepublic of Korea

Personalised recommendations