Annals of Operations Research

, Volume 271, Issue 2, pp 641–678 | Cite as

Examining military medical evacuation dispatching policies utilizing a Markov decision process model of a controlled queueing system

  • Phillip R. Jenkins
  • Matthew J. Robbins
  • Brian J. Lunday
Original Research


Military medical planners must develop dispatching policies that dictate how aerial medical evacuation (MEDEVAC) units are utilized during major combat operations. The objective of this research is to determine how to optimally dispatch MEDEVAC units in response to 9-line MEDEVAC requests to maximize MEDEVAC system performance. A discounted, infinite horizon Markov decision process (MDP) model is developed to examine the MEDEVAC dispatching problem. The MDP model allows the dispatching authority to accept, reject, or queue incoming requests based on a request’s classification (i.e., zone and precedence level) and the state of the MEDEVAC system. A representative planning scenario based on contingency operations in southern Afghanistan is utilized to investigate the differences between the optimal dispatching policy and three practitioner-friendly myopic policies. Two computational experiments are conducted to examine the impact of selected MEDEVAC problem features on the optimal policy and the system performance measure. Several excursions are examined to identify how the 9-line MEDEVAC request arrival rate and the MEDEVAC flight speeds impact the optimal dispatching policy. Results indicate that dispatching MEDEVAC units considering the precedence level of requests and the locations of busy MEDEVAC units increases the performance of the MEDEVAC system. These results inform the development and implementation of MEDEVAC tactics, techniques, and procedures by military medical planners. Moreover, an analysis of solution approaches for the MEDEVAC dispatching problem reveals that the policy iteration algorithm substantially outperforms the linear programming algorithms executed by CPLEX 12.6 with regard to computational effort. This result supports the claim that policy iteration remains the superlative solution algorithm for exactly solving computationally tractable Markov decision problems.


Markov decision processes Military medical evacuation (MEDEVAC) Admission control Queueing Emergency medical service (EMS) 



The views expressed in this paper are those of the authors and do not reflect the official policy or position of the United States Army, the United States Air Force, the Department of Defense, or the United States Government. The authors would like to thank the United States Army Medical Evacuation Proponency Directorate for its support of this research.


  1. Alsalloum, O. I., & Rand, G. K. (2006). Extensions to emergency vehicle location models. Computers & Operations Research, 33(9), 2725–2743.CrossRefGoogle Scholar
  2. Bandara, D., Mayorga, M. E., & McLay, L. A. (2012). Optimal dispatching strategies for emergency vehicles to increase patient survivability. International Journal of Operational Research, 15(2), 195–214.CrossRefGoogle Scholar
  3. Bandara, D., Mayorga, M. E., & McLay, L. A. (2014). Priority dispatching strategies for EMS systems. Journal of the Operational Research Society, 65(4), 572–587.CrossRefGoogle Scholar
  4. Bastian, N. D. (2010). A robust, multi-criteria modeling approach for optimizing aeromedical evacuation asset emplacement. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 7(1), 5–23.CrossRefGoogle Scholar
  5. Bastian, N. D., Fulton, L. V., Mitchell, R., Pollard, W., Wierschem, D., & Wilson, R. (2012). The future of vertical lift: Initial insights for aircraft capability and medical planning. Military Medicine, 177(7), 863–869.CrossRefGoogle Scholar
  6. Bixby, R. E. (2012). A brief history of linear and mixed-integer programming computation. Documenta Mathematica, Extra Volume: Optimization Stories, 107–121.Google Scholar
  7. Carter, G. M., Chaiken, J. M., & Ignall, E. (1972). Response areas for two emergency units. Operations Research, 20(3), 571–594.CrossRefGoogle Scholar
  8. Clarke, J. E., & Davis, P. R. (2012). Medical evacuation and triage of combat casualties in Helmand Province, Afghanistan: October 2010–April 2011. Military Medicine, 177(11), 1261–1266.CrossRefGoogle Scholar
  9. Cox, M. (2016). Bell touts future army helicopter design: ’V-280 is not a V-22’. Accessed 7 September 2016.
  10. De Lorenzo, R. A. (2003). Military casualty evacuation: MEDEVAC (pp. 45–59)., Aeromedical evacuation: Management of acute and stabilized patients New York: Springer.Google Scholar
  11. Department of Defense. (2016). Defense Casualties Analysis System (DCAS) Operation FREEDOM’S SENTINEL (OFS). Accessed 22 December 2016.
  12. Department of the Army. (2000). Field manual 8-10-6, medical evacuation in a theater of operations.Google Scholar
  13. Department of the Army. (2016). Army Techniques Publication 4-02.2, medical evacuation. Change 1.Google Scholar
  14. Eastridge, B. J., Mabry, R. L., Seguin, P., Cantrell, J., Tops, T., Uribe, P., et al. (2012). Death on the battlefield (2001–2011): Implications for the future of combat casualty care. Journal of Trauma and Acute Care Surgery, 73(6), S431–S437.CrossRefGoogle Scholar
  15. Fish, P. N. (2014). Army medical officer’s guide. Mechanicsburg: Stackpole Books.Google Scholar
  16. Fulton, L. V., Lasdon, L. S., McDaniel, R. R., & Coppola, M. N. (2010). Two-stage stochastic optimization for the allocation of medical assets in steady-state combat operations. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 7(2), 89–102.CrossRefGoogle Scholar
  17. Fulton, L., McMurry, P., & Kerr, B. (2009). A Monte Carlo simulation of air ambulance requirements during major combat operations. Military Medicine, 174(6), 610–614.CrossRefGoogle Scholar
  18. Garrett, M. X. (2013). USCENTCOM review of MEDEVAC procedures in Afghanistan. Technical report, United States Central Command.Google Scholar
  19. Grannan, B. C., Bastian, N. D., & McLay, L. A. (2015). A maximum expected covering problem for locating and dispatching two classes of military medical evacuation air assets. Optimization Letters, 9(8), 1511–1531.CrossRefGoogle Scholar
  20. Green, L. V., & Kolesar, P. J. (2004). Anniversary article: Improving emergency responsiveness with management science. Management Science, 50(8), 1001–1014.CrossRefGoogle Scholar
  21. Gross, D., & Harris, C. M. (1998). Fundamentals of queueing theory (4th ed.). Hoboken: Wiley.Google Scholar
  22. Hoffman, M. (2015). Army wants more adaptive HH-60 medical evacuation systems. Accessed 7 September 2016.
  23. International Council on Security and Development (ICOS). (2008). Afghanistan–Pakistan Insurgent activities in Afghanistan and Pakistan (2007). Accessed 4 January 2017.
  24. Jarvis, J. P. (1985). Approximating the equilibrium behavior of multi-server loss systems. Management Science, 31(2), 235–239.CrossRefGoogle Scholar
  25. Keneally, S. K., Robbins, M. J., & Lunday, B. J. (2016). A Markov decision process model for the optimal dispatch of military medical evacuation assets. Health Care Management Science, 19(2), 111–129.CrossRefGoogle Scholar
  26. Kotwal, R. S., Howard, J. T., Orman, J. A., Tarpey, B. W., Bailey, J. A., Champion, H. R., et al. (2016). The effect of a golden hour policy on the morbidity and mortality of combat casualties. JAMA Surgery, 151(1), 15–24.CrossRefGoogle Scholar
  27. Kuisma, M., Holmström, P., Repo, J., Määttä, T., Nousila-Wiik, M., & Boyd, J. (2004). Prehospital mortality in an EMS system using medical priority dispatching: A community based cohort study. Resuscitation, 61(3), 297–302.CrossRefGoogle Scholar
  28. Kulkarni, V. G. (2009). Modeling and analysis of stochastic systems (2nd ed.). Boca Raton: CRC Press.Google Scholar
  29. Lejeune, M. A., & Margot, F. (2016). Aeromedical battlefield evacuation under endogenous uncertainty. Technical report, Carnegie Mellon University, Pittsburg, PA.Google Scholar
  30. Leoni, R. D. (2007). Black hawk: The story of a world class helicopter. Reston: American Institute of Aeronautics.CrossRefGoogle Scholar
  31. MacFarlane, C., & Benn, C. (2003). Evaluation of emergency medical services systems: A classification to assist in determination of indicators. Emergency Medicine Journal, 20(2), 188–191.CrossRefGoogle Scholar
  32. Malsby, R. F, I. I. I., Quesada, J., Powell-Dunford, N., Kinoshita, R., Kurtz, J., Gehlen, W., et al. (2013). Prehospital blood product transfusion by US Army MEDEVAC during combat operations in Afghanistan: A process improvement initiative. Military Medicine, 178(7), 785–791.CrossRefGoogle Scholar
  33. Maxwell, M. S., Restrepo, M., Henderson, S. G., & Topaloglu, H. (2010). Approximate dynamic programming for ambulance redeployment. INFORMS Journal on Computing, 22(2), 266–281.CrossRefGoogle Scholar
  34. McLay, L. A., & Mayorga, M. E. (2010). Evaluating emergency medical service performance measures. Health Care Management Science, 13(2), 124–136.CrossRefGoogle Scholar
  35. McLay, L. A., & Mayorga, M. E. (2013). A model for optimally dispatching ambulances to emergency calls with classification errors in patient priorities. IIE Transactions, 45(1), 1–24.CrossRefGoogle Scholar
  36. Nicholl, J., Coleman, P., Parry, G., Turner, J., & Dixon, S. (1999). Emergency priority dispatch systemsa new era in the provision of ambulance services in the UK. Pre-Hospital Immediate Care, 3(2), 71–75.Google Scholar
  37. O’Shea, B. (2011). Saving lives on the battlefield. Military Medical/CBRN Technology, 15(6), 8–13.Google Scholar
  38. Powell, W. B. (2011). Approximate dynamic programming: Solving the curses of dimensionality (2nd ed.). Princeton: Wiley.CrossRefGoogle Scholar
  39. Puterman, M. L. (1994). Markov decision processes: Discrete stochastic dynamic programming. Hoboken: Wiley.CrossRefGoogle Scholar
  40. Rettke, A. J., Robbins, M. J., & Lunday, B. J. (2016). Approximate dynamic programming for the dispatch of military medical evacuation assets. European Journal of Operational Research, 254(3), 824–839.CrossRefGoogle Scholar
  41. Shenker, S., & Weinrib, A. (1989). The optimal control of heterogeneous queueing systems: A paradigm for load-sharing and routing. IEEE Transactions on Computers, 38(12), 1724–1735.CrossRefGoogle Scholar
  42. Stidham, S. (1985). Optimal control of admission to a queueing system. IEEE Transactions on Automatic Control, 30(8), 705–713.CrossRefGoogle Scholar
  43. Stidham, S, Jr. (2002). Analysis, design, and control of queueing systems. Operations Research, 50(1), 197–216.CrossRefGoogle Scholar
  44. Stidham, S, Jr., & Weber, R. (1993). A survey of Markov decision models for control of networks of queues. Queueing Systems, 13(1–3), 291–314.CrossRefGoogle Scholar
  45. Sundstrom, S. C., Blood, C. G., & Matheny, S. A. (1996). The optimal placement of casualty evacuation assets: A linear programming model. In Proceedings of the 28th conference on winter simulation (pp. 907–911). IEEE Computer Society.Google Scholar
  46. White, M. (2016). Operation ENDURING FREEDOM (OEF), Fatalities by Provinces. Accessed 22 December 2016.
  47. Zeto, J., Yamada, W., & Collins, G. (2006). Optimizing the emplacement of scarce resources to maximize the expected coverage of a geographically variant demand function. In Proceedings of technical report, US Center for Army Analysis, Ft Belvoir.Google Scholar

Copyright information

© US Government 2018

Authors and Affiliations

  1. 1.Department of Operational SciencesAir Force Institute of TechnologyWright-Patterson AFBUSA

Personalised recommendations