Skip to main content

Advertisement

SpringerLink
Log in

A simulation-based optimization approach for the calibration of a discrete event simulation model of an emergency department

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Accurate modeling of the patient flow within an Emergency Department (ED) is required by all studies dealing with the increasing and well-known problem of overcrowding. Since Discrete Event Simulation (DES) models are often adopted with the aim of assessing solutions for reducing the impact of this worldwide phenomenon, an accurate estimation of the service time of the ED processes is necessary to guarantee the reliability of the results. However, simulation models concerning EDs are frequently affected by missing data, thus requiring a proper estimation of some unknown parameters. In this paper, a simulation-based optimization approach is used to estimate the incomplete data in the patient flow within an ED by adopting a model calibration procedure. The objective function of the resulting minimization problem represents the deviation between simulation output and real data, while the constraints ensure that the response of the simulation is sufficiently accurate according to the precision required. Data from a real case study related to a big ED in Italy is used to test the effectiveness of the proposed approach. The experimental results show that the model calibration allows recovering the missing parameters, thus leading to an accurate DES model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Patients sent to OU just share with the other patients the triage, since they have a dedicated clinical pathway. Therefore, OU is not included in the downstream patient flow under study.

References

  • Ahalt, V., Argon, N., Strickler, J., & Mehrotra, A. (2018). Comparison of emergency department crowding scores: A discrete-event simulation approach. Health Care Management Science, 21, 144–155.

    Article  Google Scholar 

  • Ahmed, M. A., & Alkhamis, T. M. (2009). Simulation optimization for an emergency department healthcare unit in Kuwait. European Journal of Operational Research, 198(3), 936–942.

    Article  Google Scholar 

  • Amaran, S., Sahinidis, N., Sharda, B., & Bury, S. (2014). Simulation optimization: A review of algorithms and applications. 4OR, 12, 301–333.

    Article  Google Scholar 

  • Audet, C., Le Digabel, S., & Tribes, C. (2019). The mesh adaptive direct search algorithm for granular and discrete variables. SIAM Journal on Optimization, 29, 1164–1189. https://doi.org/10.1137/18M1175872.

    Article  Google Scholar 

  • Audet, C., & Hare, W. (2017). Derivative-free and blackbox optimization. New York: Springer. https://doi.org/10.1007/978-3-319-68913-5.

    Book  Google Scholar 

  • Baumlin, K., Shapiro, J., Weiner, C., Gottlieb, B., Chawla, N., & Richardson, L. (2010). Clinical information system and process redesign improves emergency department efficiency. The Joint Commission Journal on Quality and Patient Safety, 36(4), 179–185.

    Article  Google Scholar 

  • Bedoya-Valencia, L., & Kirac, E. (2016). Evaluating alternative resource allocation in an emergency department using discrete event simulation. Simulation, 92, 1041–1051.

    Article  Google Scholar 

  • Conn, A., Scheinber, K., & Vicente, L. N. (2009). Introduction to Derivative-Free Optimization. MPS/SIAM Book Series on Optimization, SIAM, Philadelphia.

  • De Santis, A., Giovannelli, T., Lucidi, S., Messedaglia, M., & Roma, M. (2021). Determining the optimal piecewise constant approximation for the nonhomogeneous Poisson process rate of emergency department patient arrivals. Flexible Service and Manufacturing Journal. https://doi.org/10.1007/s10696-021-09408-9.

    Article  Google Scholar 

  • Di Somma, S., Paladino, L., Vaughan, L., Lalle, I., Magrini, L., & Magnanti, M. (2014). Overcrowding in emergency department: An international issue. Internal and Emergency Medicine. https://doi.org/10.1007/s11739-014-1154-8.

    Article  Google Scholar 

  • Duma, D., & Aringhieri, R. (2018). An ad hoc process mining approach to discover patient paths of an emergency department. Flexible Services and Manufacturing Journal. https://doi.org/10.1007/s10696-018-9330-1.

    Article  Google Scholar 

  • Fava, G., Giovannelli, T., Messedaglia, M., & Roma, M. (2021). Effect of different patient peak arrivals on an emergency department via discrete event simulation: A case study. Simulation. https://doi.org/10.1177/00375497211038756.

  • Fu, M. C. (Ed.). (2015). Handbook of simulation optimization. New York: Springer.

    Google Scholar 

  • Gosavi, A. (2014). Simulation-based optimization: Parametric optimization techniques and reinforcement learning (2nd ed.). New York: Springer.

    Google Scholar 

  • Gray, G. A., & Kolda, T. G. (2006). Algorithm 856: Appspack 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Transaction on Mathematics Software, 32(3), 485–507. https://doi.org/10.1145/1163641.1163647.

    Article  Google Scholar 

  • Guo, H., Gao, S., Tsui, K., & Niu, T. (2017). Simulation optimization for medical staff configuration at emergency department in Hong Kong. IEEE Transactions on Automation Science and Engineering, 14(4), 1655–1665.

    Article  Google Scholar 

  • Guo, H., Goldsman, D., Tsui, K. L., Zhou, Y., & Wong, S. Y. (2016). Using simulation and optimisation to characterise durations of emergency department service times with incomplete data. International Journal of Production Research, 54(21), 6494–6511.

    Article  Google Scholar 

  • Hoot, N., & Aronsky, D. (2008). Systematic review of emergency department crowding: Causes, effects, and solutions. Annals of Emergency Medicine, 52(2), 126–136.

    Article  Google Scholar 

  • Joshi, V., Lim, C., & Teng, S. G. (2016). Simulation study: Improvement for non-urgent patient processes in the emergency department. Engineering Management Journal, 28(3), 145–157.

    Article  Google Scholar 

  • Kim, S., Pasupathy, R., & Henderson, S. G. (2015). A guide to sample average approximation (pp. 207–243). New York: Springer. https://doi.org/10.1007/978-1-4939-1384-8_8.

    Book  Google Scholar 

  • Kim, S. H., & Whitt, W. (2014). Are call center and hospital arrivals well modeled by nonhomogeneous Poisson process? Manufactory and Service Operations Management, 16, 464–480.

    Article  Google Scholar 

  • Kramer, A., Dosi, C., Iori, M., & Vignoli, M. (2020). Successful implementation of discrete event simulation: the case of an Italian emergency department. ArXiv arxiv:2006.13062

  • Kuo, Y. H., Rado, O., Lupia, B., Leung, J. M. Y., & Graham, C. A. (2016). Improving the efficiency of a hospital emergency department: A simulation study with indirectly imputed service-time distributions. Flexible Services and Manufacturing Journal, 28(1), 120–147.

    Article  Google Scholar 

  • Law, A. (2015). Simulation modeling and analysis (5th ed.). New York: McGraw-Hill.

    Google Scholar 

  • Liu, Z., Rexachs, D., Epelde, F., & Luque, E. (2017). A simulation and optimization based method for calibrating agent-based emergency department models under data scarcity. Computers and Industrial Engineering, 103(C), 300–309. https://doi.org/10.1016/j.cie.2016.11.036.

    Article  Google Scholar 

  • Liuzzi, G., Lucidi, S., & Rinaldi, F. (2020). An algorithmic framework based on primitive directions and nonmonotone line searches for black-box optimization problems with integer variables. Mathematical Programming Computation, 12(4), 673–702. https://doi.org/10.1007/s12532-020-00182-7.

    Article  Google Scholar 

  • Nahhas, A., Alwadi, A., & Reggelin, T. (2017). Simulation and the emergency department overcrowding problem. Procedia Engineering, 178, 368–376.

    Article  Google Scholar 

  • Pagnoncelli, B. K., Ahmed, S., & Shapiro, A. (2009). Sample average approximation method for chance constrained programming: Theory and applications. Journal of Optimization Theory and Applications, 142(2), 399–416. https://doi.org/10.1007/s10957-009-9523-6.

    Article  Google Scholar 

  • Paul, S. A., Reddy, M. C., & De Flitch, C. J. (2010). A systematic review of simulation studies investigating emergency department overcrowding. Simulation, 86(8–9), 559–571.

    Article  Google Scholar 

  • Pines, J. M., Iyer, S., Disbot, M., Hollander, J. E., Shofer, F. S., & Datner, E. M. (2008). The effect of emergency department crowding on patient satisfaction for admitted patients. Academic Emergency Medicine, 15(9), 825–831. https://doi.org/10.1111/j.1553-2712.2008.00200.x.

    Article  Google Scholar 

  • Salmon, A., Rachuba, S., Briscoe, S., & Pitt, M. (2018). A structured literature review of simulation modelling applied to emergency departments: Current patterns and emerging trends. Operations Research for Health Care, 19, 1–13.

    Article  Google Scholar 

  • Ucar, I., Smeets, B., & Azcorra, A. (2019). Simmer: Discrete-event simulation for R. Journal of Statistical Software, 90(2), 1–30. https://doi.org/10.18637/jss.v090.i02.

    Article  Google Scholar 

  • Vanbrabant, L., Martin, N., Ramaekers, K., & Braekers, K. (2019). Quality of input data in emergency department simulations: Framework and assessment techniques. Simulation Modelling Practice and Theory, 91, 83–101.

    Article  Google Scholar 

  • Whitt, W., & Zhang, X. (2017). A data-driven model of an emergency department. Operations Research for Health Care, 12, 1–15.

    Article  Google Scholar 

  • Wiler, J., Griffey, R., & Olsen, T. (2011). Review of modeling approaches for emergency department patient flow and crowding research. Academic Emergency Medicine, 18, 1371–1379.

    Article  Google Scholar 

  • Wong, Z.S.Y., Lit, A.C.H., Leung, S.Y., Tsui, K.L., & Chin, K.S. (2016). A discrete-event simulation study for emergency room capacity management in a Hong Kong hospital. 2016 Winter Simulation Conference (WSC) pp. 1970–1981

  • Yousefi, M., Yousefi, M., & Fogliatto, F. S. (2020). Simulation-based optimization methods applied in hospital emergency departments: A systematic review. SIMULATION, 96(10), 791–806. https://doi.org/10.1177/0037549720944483.

    Article  Google Scholar 

  • Zeinali, F., Mahootchi, M., & Sepehri, M. (2015). Resource planning in the emergency departments: A simulation-base metamodeling approach. Simulation Modelling Practice and Theory, 53, 123–138.

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Prof. F. Romano (Chief Medical Officer) and Dr. L. De Vito (Medical Director of ED) of Policlinico Umberto I of Rome for their availability to carry out this study. Moreover, the authors are really thankful to the anonymous Referees for their comments and fruitful suggestions which led to improving very much the paper. In particular, they take the chance to express their gratitude to a Reviewer for his/her thorough and sharp work.

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria Informatica Automatica e Gestionale “A. Ruberti”, SAPIENZA Università di Roma, via Ariosto, 25, 00185, Roma, Italy

    Alberto De Santis, Stefano Lucidi & Massimo Roma

  2. ACTOR Start up of SAPIENZA Università di Roma, via Nizza 45, 00198, Roma, Italy

    Mauro Messedaglia

  3. Department of Industrial and Systems Engineering, Lehigh University, 200 West Packer Avenue, Bethlehem, PA, 18015-1582, USA

    Tommaso Giovannelli

Authors
  1. Alberto De Santis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tommaso Giovannelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Lucidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mauro Messedaglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Massimo Roma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Massimo Roma.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Model calibration: Plots I

This appendix focuses on the KPI given by the average hourly number of patients satisfying two conditions: (i) they have already started the medical visit; (ii) they have not been discharged yet. In particular, the plots report the comparison between the value of the KPI computed through the real data and the corresponding value derived from the simulation output along with its \(95\%\) confidence interval. The comparisons are performed for all color tags \(c \in C\) and for all the ED units \(u \in U(c)\) where a patient with color tag c can be assigned.

figure a
figure b
figure c
figure d
figure e
figure f
figure g
figure h

Model calibration: Plots II

This appendix reports the comparison between the Empirical Cumulative Distribution Functions (ECDFs) of the time differences DOT and DIT computed through the real data and the simulation output resulting from the calibration procedure. The comparisons are performed for all color tags \(c \in C\) and for all ED units \(u \in U(c)\). The colored curves correspond to the simulation output.

figure i
figure j
figure k
figure l
figure m
figure n
figure o
figure p

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Santis, A., Giovannelli, T., Lucidi, S. et al. A simulation-based optimization approach for the calibration of a discrete event simulation model of an emergency department. Ann Oper Res 320, 727–756 (2023). https://doi.org/10.1007/s10479-021-04382-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-021-04382-9

Keywords

Search

Navigation