Skip to main content

Predicting and Assessing Wildfire Evacuation Decision-Making Using Machine Learning: Findings from the 2019 Kincade Fire

Fire Technology volume 59, pages 793–825 (2023)Cite this article

Abstract

To develop effective wildfire evacuation plans, it is crucial to study evacuation decision-making and identify the factors affecting individuals’ choices. Statistic models (e.g., logistic regression) are widely used in the literature to predict household evacuation decisions, while the potential of machine learning models has not been fully explored. This study compared seven machine learning models with logistic regression to identify which approach is better for predicting a householder’s decision to evacuate. The machine learning models tested include the naïve Bayes classifier, K-nearest neighbors, support vector machine, neural network, classification and regression tree (CART), random forest, and extreme gradient boosting. These models were calibrated using the survey data collected from the 2019 Kincade Fire. The predictive performance of the machine learning models and the logistic regression was compared using F1 score, accuracy, precision, and recall. The results indicate that all the machine learning models performed better than the logistic regression. The CART model has the highest F1 score among all models, with a statistically significant difference from the logistic regression model. This CART model shows that the most important factor affecting the decision to evacuate is pre-fire safety perception. Other important factors include receiving an evacuation order, household risk perception (during the event), and education level.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6

Notes

  1. Risk perception (and other ‘latent’ variables) are not necessarily an independent variable per se, but instead, a mediator variable when estimating evacuation decisions and movement. In future work, we hope to create a model which is ‘dependent’ only on the factors that can be obtained from any given affected area (e.g., population, environmental, and place-based variables)—to then model threat and risk perceptions and then the evacuation decision.

References

  1. Bagloee SA, Johansson KH, Asadi M (2019) A hybrid machine-learning and optimization method for contraflow design in post-disaster cases and traffic management scenarios. Expert Syst Appl 124:67–81. https://doi.org/10.1016/j.eswa.2019.01.042

    Article  Google Scholar 

  2. Bandini S, Manzoni S, Mauri G, et al (2008) Gp generation of pedestrian behavioral rules in an evacuation model based on sca. In: International Conference on Cellular Automata, Springer, pp 409–416, https://doi.org/10.1007/978-3-540-79992-4_53

  3. Benight C, Gruntfest E, Sparks K (2004) Colorado wildfires 2002. Quick response rep. 167. Natural Hazards Center, University of Colorado Boulder.

  4. Boustras G, Ronchi E, Rein G (2017) Fires: fund research for citizen safety. Nature 551(7680):300–301. https://doi.org/10.1038/d41586-017-06020-6

    Article  Google Scholar 

  5. Bowman D, Williamson G, Yebra M et al (2020) Wildfires: Australia needs national monitoring agency. Nature. https://doi.org/10.1038/d41586-020-02306-4

    Article  Google Scholar 

  6. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  7. Bzdok D, Krzywinski M, Altman N (2018) Machine learning: supervised methods. Nat Methods 15(1):5. https://doi.org/10.1038/nmeth.4551

    Article  Google Scholar 

  8. Chen H, Chen H, Liu Z et al (2020) Analysis of factors affecting the severity of automated vehicle crashes using XGBoost model combining POI data. J Adv Trans. https://doi.org/10.1155/2020/8881545

    Article  Google Scholar 

  9. Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 785–794, https://doi.org/10.1145/2939672.2939785

  10. Chen T, He T, Benesty M et al (2019) Package ‘xgboost’. R version 90. https://doi.org/10.1145/2939672.2939785

  11. Cheng L, Chen X, De Vos J et al (2019) Applying a random forest method approach to model travel mode choice behavior. Travel Behav Soc 14:1–10. https://doi.org/10.1016/j.tbs.2018.09.002

    Article  Google Scholar 

  12. Chinchor N (1992) Muc-4 evaluation metrics in proc. of the fourth message understanding conference. pp. 22–29

  13. Devos O, Ruckebusch C, Durand A et al (2009) Support vector machines (svm) in near infrared (nir) spectroscopy: focus on parameters optimization and model interpretation. Chemometrics Intell Lab Syst 96(1):27–33. https://doi.org/10.1016/j.chemolab.2008.11.005

    Article  Google Scholar 

  14. Eriksen C, Gill N, Head L (2010) The gendered dimensions of bushfire in changing rural landscapes in Australia. J Rural Stud 26(4):332–342. https://doi.org/10.1016/j.jrurstud.2010.06.001

    Article  Google Scholar 

  15. Fischer HW, Stine GF, Stoker BL, et al (1995) Evacuation behaviour: Why do some evacuate, while others do not? A case study of the Ephrata, Pennsylvania (USA) evacuation. Disaster Prevent Mana: Int J https://doi.org/10.1108/09653569510093414

  16. Fix E, Hodges JL (1989) Discriminatory analysis. Nonparametric discrimination: consistency properties. Int Stat Rev/Revue Internationale de Statistique 57(3):238–247. https://doi.org/10.2307/1403797

    Article  MATH  Google Scholar 

  17. Folk LH, Kuligowski ED, Gwynne S et al (2019) A provisional conceptual model of human behavior in response to wildland-urban interface fires. Fire Technol 55(5):1619–1647. https://doi.org/10.1007/s10694-019-00821-z

    Article  Google Scholar 

  18. Forcael E, González V, Orozco F et al (2014) Ant colony optimization model for tsunamis evacuation routes. Comput Aided Civil Inf Eng 29(10):723–737. https://doi.org/10.1111/mice.12113

    Article  Google Scholar 

  19. Hagenauer J, Helbich M (2017) A comparative study of machine learning classifiers for modeling travel mode choice. Expert Syst Appl 78:273–282. https://doi.org/10.1016/j.eswa.2017.01.057

    Article  Google Scholar 

  20. Inkoom S, Sobanjo J, Barbu A et al (2019) Pavement crack rating using machine learning frameworks: Partitioning, bootstrap forest, boosted trees, naïve bayes, and k-nearest neighbors. J Trans Eng B Pavements 145(3):04019,031. https://doi.org/10.1061/JPEODX.0000126

    Article  Google Scholar 

  21. James G, Witten D, Hastie T et al (2013) An introduction to statistical learning, vol 112. Springer, Berlin. https://doi.org/10.1007/978-1-0716-1418-1_1

    Book  MATH  Google Scholar 

  22. Komura D, Ishikawa S (2018) Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 16:34–42. https://doi.org/10.1016/j.csbj.2018.01.001

    Article  Google Scholar 

  23. Kuhn M (2008) Building predictive models in R using the caret package. J Stat Softw 28:1–26. https://doi.org/10.18637/jss.v028.i05

    Article  Google Scholar 

  24. Kuligowski ED, Walpole EH, Lovreglio R et al (2020) Modelling evacuation decision-making in the 2016 Chimney Tops 2 fire in Gatlinburg. TN. Int J Wildland Fire 29(12):1120–1132. https://doi.org/10.1071/WF20038

    Article  Google Scholar 

  25. Kuligowski ED, Zhao X, Lovreglio R et al (2022) Modeling evacuation decisions in the 2019 Kincade fire in California. Saf Sci 146(105):541. https://doi.org/10.1016/j.ssci.2021.105541

    Article  Google Scholar 

  26. Lamounier E, Soares A, Andrade A, et al (2002) A virtual prosthesis control based on neural networks for emg pattern classification. In: Proceedings of the Artificial Intelligence and Soft Computing, Citeseer

  27. Lewis RJ (2000) An introduction to classification and regression tree (cart) analysis. In: Annual meeting of the society for academic emergency medicine in San Francisco, California, Citeseer

  28. Lhéritier A, Bocamazo M, Delahaye T et al (2019) Airline itinerary choice modeling using machine learning. J Choice Modell 31:198–209. https://doi.org/10.1016/j.jocm.2018.02.002

    Article  Google Scholar 

  29. Liaw A, Wiener M et al (2002) Classification and regression by randomforest. R News 2(3):18–22

    Google Scholar 

  30. Lindell MK, Perry RW (2012) The protective action decision model: theoretical modifications and additional evidence. Risk Anal: Int J 32(4):616–632. https://doi.org/10.1111/j.1539-6924.2011.01647.x

    Article  Google Scholar 

  31. Liu M, Lo SM (2011) The quantitative investigation on people’s pre-evacuation behavior under fire. Autom Constr 20(5):620–628. https://doi.org/10.1016/j.autcon.2010.12.004

    Article  Google Scholar 

  32. Lo S, Liu M, Zhang P et al (2009) An artificial neural-network based predictive model for pre-evacuation human response in domestic building fire. Fire Technol 45(4):431–449. https://doi.org/10.1007/s10694-008-0064-6

    Article  Google Scholar 

  33. Lopez C, Marti JR, Sarkaria S (2018) Distributed reinforcement learning in emergency response simulation. IEEE Access 6:67,261-67,276. https://doi.org/10.1109/ACCESS.2018.2878894

    Article  Google Scholar 

  34. Lovreglio R, Kuligowski E, Gwynne S et al (2019) A modelling framework for householder decision-making for wildfire emergencies. Int J Disaster Risk Reduct 41(101):274. https://doi.org/10.1016/j.ijdrr.2019.101274

    Article  Google Scholar 

  35. Lovreglio R, Kuligowski E, Walpole E et al (2020) Calibrating the wildfire decision model using hybrid choice modelling. Int J Disaster Risk Reduct 50(101):770. https://doi.org/10.1016/j.ijdrr.2020.101770

    Article  Google Scholar 

  36. McCaffrey S, Wilson R, Konar A (2018) Should I stay or should I go now? or should I wait and see? Influences on wildfire evacuation decisions. Risk Anal 38(7):1390–1404. https://doi.org/10.1111/risa.12944

    Article  Google Scholar 

  37. McCaffrey SM, Winter G (2011) Understanding homeowner preparation and intended actions when threatened by a wildfire. Proceedings of the Second Conference on the Human Dimensions of Wildland Fire

  38. McCallum A, Nigam K, et al (1998) A comparison of event models for naive bayes text classification. In: AAAI-98 workshop on learning for text categorization, Citeseer, pp 41–48

  39. McLennan J (2014) Capturing community members’ bushfire experiences: Interviews with residents following the 12 January 2014 Parkerville (WA) fire

  40. McLennan J, Elliott G, Omodei M (2011) Issues in community bushfire safety: analyses of interviews conducted by the 2009 Victorian bushfires research task force. Bundoora, AU

    Google Scholar 

  41. McLennan J, Elliott G, Omodei M (2012) Householder decision-making under imminent wildfire threat: stay and defend or leave? Int J Wildland Fire 21(7):915–925. https://doi.org/10.1071/WF11061

    Article  Google Scholar 

  42. McLennan J, Elliott G, Omodei M et al (2013) Householders’ safety-related decisions, plans, actions and outcomes during the 7 February 2009 Victorian (Australia) wildfires. Fire Saf J 61:175–184. https://doi.org/10.1016/j.firesaf.2013.09.003

    Article  Google Scholar 

  43. McLennan J, Paton D, Beatson R (2015) Psychological differences between south-eastern australian householders’ who intend to leave if threatened by a wildfire and those who intend to stay and defend. Int J Disaster Risk Reduct 11:35–46. https://doi.org/10.1016/j.ijdrr.2014.11.008

    Article  Google Scholar 

  44. McNeill IM, Dunlop PD, Skinner TC et al (2016) A value-and expectancy-based approach to understanding residents’ intended response to a wildfire threat. Int J Wildland Fire 25(4):378–389. https://doi.org/10.1071/WF15051

    Article  Google Scholar 

  45. Meyer D, Dimitriadou E, Hornik K, et al (2019) Package ‘e1071’. The R Journal

  46. Molnar C (2020) Interpretable machine learning. Lulu. com, https://christophm.github.io/interpretable-ml-book/

  47. Mozumder P, Raheem N, Talberth J et al (2008) Investigating intended evacuation from wildfires in the wildland-urban interface: application of a bivariate probit model. For Policy Econ 10(6):415–423. https://doi.org/10.1016/j.forpol.2008.02.002

    Article  Google Scholar 

  48. Nakagawa S (2004) A farewell to Bonferroni: the problems of low statistical power and publication bias. Behav Ecol 15(6):1044–1045. https://doi.org/10.1093/beheco/arh107

    Article  Google Scholar 

  49. National Interagency Fire Center (2022) Wildland fire statistics. https://www.nifc.gov/fireInfo/fireInfo_statistics.html. Accessed March 10, 2022

  50. Ng A, Jordan M (2001) On discriminative vs. generative classifiers: a comparison of logistic regression and naive Bayes. Advances in Neural Information Processing Systems 14

  51. Paveglio T, Prato T, Dalenberg D et al (2014) Understanding evacuation preferences and wildfire mitigations among Northwest Montana residents. Int J Wildland Fire 23(3):435–444. https://doi.org/10.1071/WF13057

    Article  Google Scholar 

  52. Perneger T (2014) What’s wrong with Bonferroni adjustments. BMJ 316(7139):1236–1238. https://doi.org/10.1136/bmj.316.7139.1236

    Article  Google Scholar 

  53. Radeloff VC, Helmers DP, Kramer HA et al (2018) Rapid growth of the us wildland-urban interface raises wildfire risk. Proc Natl Acad Sci 115(13):3314–3319. https://doi.org/10.1073/pnas.1718850115

    Article  Google Scholar 

  54. Ripley B, Ripley MB (2016) Package ‘tree’. Classification and Regression Trees Version. pp. 1–0

  55. Ripley B, Venables W, Ripley MB (2015) Package ‘class’. The Comprehensive R Archive Network. p 11

  56. Ripley B, Venables W, Ripley M (2016) Package ‘nnet’r package version, 7:3–12

  57. Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1(5):206–215

    Article  Google Scholar 

  58. Şahin C, Rokne J, Alhajj R (2019) Human behavior modeling for simulating evacuation of buildings during emergencies. Phys A: Stat Mech Appl 528(121):432. https://doi.org/10.1016/j.physa.2019.121432

    Article  Google Scholar 

  59. Sharma S, Singh H, Prakash A (2008) Multi-agent modeling and simulation of human behavior in aircraft evacuations. IEEE Trans Aerospace Electron Syst 44(4):1477–1488. https://doi.org/10.1109/TAES.2008.4667723

    Article  Google Scholar 

  60. Song X, Zhang Q, Sekimoto Y, et al (2013) Modeling and probabilistic reasoning of population evacuation during large-scale disaster. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1231–1239, https://doi.org/10.1145/2487575.2488189

  61. Song YY, Ying L (2015) Decision tree methods: applications for classification and prediction. Shanghai Arch Psychiatr 27(2):130

    Google Scholar 

  62. Stasiewicz AM, Paveglio TB (2021) Preparing for wildfire evacuation and alternatives: exploring influences on residents’ intended evacuation behaviors and mitigations. Int J Disaster Risk Reduct 58(102):177. https://doi.org/10.1016/j.ijdrr.2021.102177

    Article  Google Scholar 

  63. Strahan KW, Whittaker J, Handmer J (2019) Predicting self-evacuation in Australian bushfire. Environ Hazards 18(2):146–172. https://doi.org/10.1080/17477891.2018.1512468

    Article  Google Scholar 

  64. Strawderman L, Salehi A, Babski-Reeves K et al (2012) Reverse 911 as a complementary evacuation warning system. Nat Hazards Rev 13(1):65–73. https://doi.org/10.1061/(ASCE)NH.1527-6996.0000059

    Article  Google Scholar 

  65. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press, Cambridge

    MATH  Google Scholar 

  66. Tang P, Shen GQ (2015) Decision-making model to generate novel emergency response plans for improving coordination during large-scale emergencies. Knowl Based Syst 90:111–128. https://doi.org/10.1016/j.knosys.2015.09.027

    Article  Google Scholar 

  67. Toledo T, Marom I, Grimberg E et al (2018) Analysis of evacuation behavior in a wildfire event. Int J Disaster Risk Reduct 31:1366–1373. https://doi.org/10.1016/j.ijdrr.2018.03.033

    Article  Google Scholar 

  68. Wang K, Shi X, Goh APX et al (2019) A machine learning based study on pedestrian movement dynamics under emergency evacuation. Fire Saf J 106:163–176. https://doi.org/10.1016/j.firesaf.2019.04.008

    Article  Google Scholar 

  69. Whittaker J, Handmer J (2010) Review of key bushfire research findings. Report Number WIT 3007:0041

    Google Scholar 

  70. Whittaker J, Eriksen C, Haynes K (2015) More men die in bushfires: how gender affects how we plan and respond. The Conversation

  71. Whittaker J, Eriksen C, Haynes K (2016) Gendered responses to the 2009 black saturday bushfires in Victoria, Australia. Geograph Res 54(2):203–215. https://doi.org/10.1111/1745-5871.12162

    Article  Google Scholar 

  72. Wong SD (2020) Compliance, congestion, and social equity: tackling critical evacuation challenges through the sharing economy, joint choice modeling, and regret minimization. University of California, Berkeley

    Google Scholar 

  73. Wong SD, Broader JC, Shaheen SA (2020) Review of California wildfire evacuations from 2017 to 2019. https://doi.org/10.7922/G29G5K2R

  74. Wong SD, Broader JC, Walker JL et al (2022) Understanding California wildfire evacuee behavior and joint choice making. Transportation. https://doi.org/10.1007/s11116-022-10275-y

    Article  Google Scholar 

  75. Wu A, Yan X, Kuligowski E et al (2022) Wildfire evacuation decision modeling using GPS data. Int J Disaster Risk Reduct 83:103373. https://doi.org/10.1016/j.ijdrr.2022.103373

    Article  Google Scholar 

  76. Xie C, Lu J (1854) Parkany E (2003) Work travel mode choice modeling with data mining: decision trees and neural networks. Transport Res Record 1:50–61. https://doi.org/10.3141/1854-06

    Article  Google Scholar 

  77. Xu Y, Yan X, Liu X et al (2021) Identifying key factors associated with ridesplitting adoption rate and modeling their nonlinear relationships. Trans Res A: Policy Prac 144:170–188. https://doi.org/10.1016/j.tra.2020.12.005

    Article  Google Scholar 

  78. Zhao X, Lovreglio R, Nilsson D (2020) Modelling and interpreting pre-evacuation decision-making using machine learning. Autom Constr 113:103140. https://doi.org/10.1016/j.autcon.2020.103140

    Article  Google Scholar 

  79. Zhao X, Lovreglio R, Kuligowski E et al (2021) Using artificial intelligence for safe and effective wildfire evacuations. Fire Technol 57(2):483–485. https://doi.org/10.1007/s10694-020-00979-x

    Article  Google Scholar 

  80. Zhao X, Xu N, Yang K et al (2021) Modeling evacuation behavior in the 2019 Kincade Fire, Sonoma County, California. Natural Hazards Center Quick Response Grant Report Series, 326. Boulder, CO: Natural Hazards Center, University of Colorado Boulder. Available at: https://hazards.colorado.edu/quick-response-report/modeling-evacuation-behavior-in-the-2019-kincade-fire-sonoma-county-california

  81. Zhao X, Xu Y, Lovreglio R et al (2022) Estimating wildfire evacuation decision and departure timing using large-scale GPS data. Trans Res D: Trans Environ 107:103277. https://doi.org/10.1016/j.trd.2022.103277

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the Natural Hazards Center Quick Response Research Program. The Quick Response program is based on work supported by the National Science Foundation (Award #1635593). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF or the Natural Hazards Center. The authors would like to thank the many residents of Sonoma County, California for sharing their experiences with us for this study.

Author information

Authors and Affiliations

  1. Department of Civil and Coastal Engineering, University of Florida, 1949 Stadium Rd, Gainesville, FL, 32611, USA

    Ningzhe Xu & Xilei Zhao

  2. Department of Civil, Construction and Environmental Engineering, The University of Alabama, Tuscaloosa, USA

    Ningzhe Xu

  3. School of Built Environment, Massey University, Auckland, New Zealand

    Ruggiero Lovreglio

  4. School of Engineering, RMIT University, Melbourne, Australia

    Erica D. Kuligowski

  5. Department of Geography, University of Utah, Salt Lake City, USA

    Thomas J. Cova

  6. Department of Civil and Natural Resources Engineering, University of Canterbury, Christchurch, New Zealand

    Daniel Nilsson

Authors
  1. Ningzhe Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruggiero Lovreglio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erica D. Kuligowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas J. Cova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Nilsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xilei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xilei Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: The Results of Logistic Regression

See Table 4

Table 4 The Results of Logistic Regression
Full size table

Appendix B: The Results of Machine Learning Models

See Table 5

Table 5 Variable Importance for Each Machine Learning Model (top 10)
Full size table

Appendix C: Paired t-test Results with the Bonferroni Correction

See Tables 6, 7

Table 6 Paired t-test Results (p-value) of F1 Score with the Bonferroni Correction (i.e., Correcting p-value by Multiplying the Number of Comparisons)
Full size table
Table 7 Paired t-test Results (p-value) of Accuracy with the Bonferroni Correction (i.e., Correcting p-value by Multiplying the Number of Comparisons)
Full size table

Appendix D: Source Code

The model comparison and paired t-test programs were implemented in R and open-sourced on GitHub (https://github.com/EvacuationBehavior/ML-for-Modeling-Wildfire-Eva-DM).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xu, N., Lovreglio, R., Kuligowski, E.D. et al. Predicting and Assessing Wildfire Evacuation Decision-Making Using Machine Learning: Findings from the 2019 Kincade Fire. Fire Technol 59, 793–825 (2023). https://doi.org/10.1007/s10694-023-01363-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10694-023-01363-1

Keywords

  • Evacuation
  • Machine learning
  • Decision-making
  • Wildfire