Internal and external validation of vulnerability indices: a case study of the Multivariate Nursing Home Vulnerability Index

  • Matthew Wilson
  • Sandi Lane
  • Raghuveer Mohan
  • Margaret SuggEmail author
Original Paper


As the frequency of natural disasters increases, there has been an emphasis on vulnerability index creation studies. In this study, we test the validity of vulnerability indices by examining a vulnerability index created for nursing homes throughout the Southeastern United States. In this index, underlying community characteristics, natural hazards frequency, and nursing home facility data were combined to create the Multivariate Nursing Home Vulnerability Index (MNHVI) using an inductive-hierarchical index structure. To internally validate these indices, a manual construction method and Monte Carlo simulations are used to create multiple unique versions of the MNHVI. Each iteration of the MNHVI considers alternative model structures for insight into regions of precision within the model and the average amount of variation for each census unit. External validation is used to determine if the indices are accurately predicting harm or mortality caused by storm events. External validation was assessed using death and injury statistics from NOAA’s storm event database and North Carolina death certificate data. Results demonstrated that indices were not precise despite changes in spatial scale and that hazard level indices were the most accurate predictor of injury and death from natural hazards. Identifying accuracy and precision for vulnerability indices provides additional assurance on the appropriate identification of at-risk regions.


Vulnerability index Social vulnerability Natural hazards Nursing homes Validation 


Supplementary material

11069_2019_3837_MOESM1_ESM.docx (321 kb)
Supplementary material 1 (DOCX 321 kb)


  1. Adger WN, Hughes TP, Folke C, Carpenter SR, Rockström J (2005) Social-ecological resilience to coastal disasters. Science 309(5737):1036–1039CrossRefGoogle Scholar
  2. Alonso JA, Lamata MT (2006) Consistency in the analytic hierarchy process: a new approach. Int J Uncertain fuzziness Knowl-Based Syst 14(04):445–459CrossRefGoogle Scholar
  3. Andersen LM, Sugg MM (2019) Geographic multi-criteria evaluation and validation: a case study of wildfire vulnerability in Western North Carolina, USA following the 2016 wildfires. Int J Disaster Risk Reduct 39:101–123CrossRefGoogle Scholar
  4. Anderson B, Chen Z (2017) Noaastormevents: explore NOAA storm events database. R package version 0.1.0. Accessed 28 Mar 2019
  5. Anderson B, Schumacher A, Crosson W, Al-Hamdan M, Yan M, Ferreri J, Chen Z, Quiring S, Guikema S (2017) Hurricaneexposuredata: data characterizing exposure to hurricanes in united states counties. R package version 0.0.2. Accessed 28 Mar 2019
  6. Andrew MK, Mitnitski AB, Rockwood K (2008) Social vulnerability, frailty and mortality in elderly people. PLoS ONE 3(5):e2232CrossRefGoogle Scholar
  7. Balica SF, Douben N, Wright NG (2009) Flood vulnerability indices at varying spatial scales. Water Sci Technol 60(10):2571–2580CrossRefGoogle Scholar
  8. Berrouet LM, Machado J, Villegas-Palacio C (2018) Vulnerability of socio—ecological systems: a conceptual framework. Ecol Ind 84:632–647CrossRefGoogle Scholar
  9. Booysen F (2002) An overview and evaluation of composite indices of development. Soc Indic Res 59(2):115–151CrossRefGoogle Scholar
  10. Brown CE (1998) Coefficient of variation. In: Applied multivariate statistics in geohydrology and related sciences. Springer, Berlin, pp 155–157CrossRefGoogle Scholar
  11. Brown L, Rothman M, Norris F (2007) Issues in mental health care for older adults after disasters. Generations 31(4):21–26Google Scholar
  12. Brunkard J, Namulanda G, Ratard R (2008) Hurricane katrina deaths, louisiana, 2005. Disaster Med Public Health Prep 2(4):215–223CrossRefGoogle Scholar
  13. CDC (2002) Instruction manual [PDF file]. Retrieved from, Accessed 28 Mar 2019
  14. Chakraborty J, Tobin GA, Montz BE (2005) Population evacuation: assessing spatial variability in geophysical risk and social vulnerability to natural hazards. Nat Hazards Rev 6(1):23–33CrossRefGoogle Scholar
  15. Chen Y, Yu J, Khan S (2010) Spatial sensitivity analysis of multi-criteria weights in GIS-based land suitability evaluation. Environ Model Softw 25(12):1582–1591CrossRefGoogle Scholar
  16. CMS (2019) Nursing Home Compare database. Retrieved from Accessed 28 Mar 2019
  17. Crosetto M, Tarantola S (2001) Uncertainty and sensitivity analysis: tools for GIS-based model implementation. Int J Geogr Inf Sci 15(5):415–437CrossRefGoogle Scholar
  18. Cutter SL, Emrich CT (2006) Moral hazard, social catastrophe: the changing face of vulnerability along the hurricane coasts. Ann Am Acad Polit Soc Sci 604(1):102–112CrossRefGoogle Scholar
  19. Cutter SL, Finch C (2008) Temporal and spatial changes in social vulnerability to natural hazards. Proc Natl Acad Sci 105(7):2301–2306CrossRefGoogle Scholar
  20. Cutter SL, Boruff BJ, Shirley WL (2003) Social vulnerability to environmental hazards. Soc Sci Q 84(2):242–261CrossRefGoogle Scholar
  21. Cutter, S. L., Gall, M., & Emrich, C. T. (2008). Toward a comprehensive loss inventory of weather and climate hazards. In: Climate extremes and society, pp 279–295Google Scholar
  22. DHHS (2016) Medicare and medicaid programs; emergency preparedness requirements for medicare and medicaid participating providers and suppliers; final rule. Federal Regist 81(180), 63859–64044. Accessed 28 Mar 2019
  23. Dosa D, Hyer K, Thomas K, Swaminathan S, Feng Z, Brown L, Mor V (2012) To evacuate or shelter in place: implications of universal hurricane evacuation policies on nursing home residents. J Am Med Dir As 13(2):190-e1Google Scholar
  24. Emrich CT, Cutter SL (2011) Social vulnerability to climate-sensitive hazards in the southern United States. Weather Clim Soc 3(3):193–208CrossRefGoogle Scholar
  25. Evans J (2010) Mapping the vulnerability of older persons to disasters. Int J Older People Nurs 5(1):63–70CrossRefGoogle Scholar
  26. Feizizadeh B, Kienberger S (2017) Spatially explicit sensitivity and uncertainty analysis for multicriteria-based vulnerability assessment. J Environ Plann Manag 60(11):2013–2035CrossRefGoogle Scholar
  27. Feizizadeh B, Jankowski P, Blaschke T (2014) A GIS based spatially-explicit sensitivity and uncertainty analysis approach for multi-criteria decision analysis. Comput Geosci 64:81–95CrossRefGoogle Scholar
  28. FEMA (2018) 2018–2022 strategic plan. Helping people. Together. Accessed 28 Mar 2019
  29. Flanagan BE, Gregory EW, Hallisey EJ, Heitgerd JL, Lewis B (2011) A social vulnerability index for disaster management. J Homel Secur Emerg Manag 8(1):1–22Google Scholar
  30. Freedman A (2012) Hurricane Irene ranked most costly category 1 storm. Accessed 28 Mar 2019
  31. Füssel HM (2007) Vulnerability: a generally applicable conceptual framework for climate change research. Global Environ Change 17(2):155–167CrossRefGoogle Scholar
  32. Galea S, Ahern J, Karpati A (2005) A model of underlying socioeconomic vulnerability in human populations: evidence from variability in population health and implications for public health. Soc Sci Med 60(11):2417–2430CrossRefGoogle Scholar
  33. Gall M (2007) Indices of social vulnerability to natural hazards: a comparative evaluation. University of South Carolina, Ann Arbor, p 231Google Scholar
  34. Gall M, Borden KA, Cutter SL (2009) When do losses count? Six fallacies of natural hazards loss data. Bull Am Meteor Soc 90(6):799–810CrossRefGoogle Scholar
  35. Greene R, Devillers R, Luther JE, Eddy BG (2011) GIS-based multiple- criteria decision analysis. Geogr Compass 5(6):412–432CrossRefGoogle Scholar
  36. Haines A, Kovats RS, Campbell-Lendrum D, Corvalán C (2006) Climate change and human health: impacts, vulnerability and public health. Public health 120(7):585–596CrossRefGoogle Scholar
  37. Hames E, Stoler J, Emrich CT, Tewary S, Pandya N (2016) A GIS approach to identifying socially and medically vulnerable older adult populations in South Florida. The Gerontologist 57(6):1133–1141Google Scholar
  38. HIFLD (2018) Homeland Infrastructure Foundation-level data. Accessed 28 Mar 2019
  39. Kahle D, Wickham H (2013) Ggmap: spatial visualization with ggplot2. R Journal 5(1):144–161CrossRefGoogle Scholar
  40. Knowlton K, Rotkin-Ellman M, King G, Margolis HG, Smith D, Solomon G, English P (2008) The 2006 California heat wave: impacts on hospitalizations and emergency department visits. Environ Health Perspect 117(1):61–67CrossRefGoogle Scholar
  41. Laditka SB, Laditka JN, Xirasagar S, Cornman CB, Davis CB, Richter JV (2008) Providing shelter to nursing home evacuees in disasters: lessons from Hurricane Katrina. Am J Public Health 98(7):1288–1293CrossRefGoogle Scholar
  42. MacEachren AM, Kraak MJ (2001) Research challenges in geovisualization. Cartogr Geogr Inf Sci 28(1):3–12CrossRefGoogle Scholar
  43. Maercker A, Brewin CR, Bryant RA, Cloitre M, van Ommeren M, Jones LM, Somasundaram DJ (2013) Diagnosis and classification of disorders specifically associated with stress: proposals for ICD-11. World Psychiatry 12(3):198–206CrossRefGoogle Scholar
  44. Malik S, Lee DC, Doran KM, Grudzen CR, Worthing J, Portelli I, Smith SW (2018) Vulnerability of older adults in disasters: emergency department utilization by geriatric patients after Hurricane Sandy. Disaster Med Public Health Prep 12(2):184–193CrossRefGoogle Scholar
  45. McLaughlin P, Dietz T (2008) Structure, agency and environment: toward an integrated perspective on vulnerability. Glob Environ Change 18(1):99–111CrossRefGoogle Scholar
  46. McLaughlin S, McKenna J, Cooper JAG (2002) Socio-economic data in coastal vulnerability indices: constraints and opportunities. J Coastal Res 36(sp1):487–497CrossRefGoogle Scholar
  47. Montz BE, Tobin GA (2011) Natural hazards: an evolving tradition in applied geography. Appl Geogr 31(1):1–4CrossRefGoogle Scholar
  48. Morath DP (2010) Social vulnerability and public health: developing a metric for medical emergency management in Florida. A thesis presented for the degree of Master of Science at the University of South CarolinaGoogle Scholar
  49. Morris JN, Fries BE, Mehr DR, Hawes C, Phillips C, Mor V, Lipsitz LA (1994) MDS cognitive performance scale©. J Gerontol 49(4):M174–M182CrossRefGoogle Scholar
  50. Morris JN, Fries BE, Morris SA (1999) Scaling ADLs within the MDS. J Gerontol Ser A 54(11):M546–M553CrossRefGoogle Scholar
  51. Nguyen TT, Bonetti J, Rogers K, Woodroffe CD (2016) Indicator-based assessment of climate-change impacts on coasts: a review of concepts, methodological approaches and vulnerability indices. Ocean Coast Manag 123:18–43CrossRefGoogle Scholar
  52. NHC. (2017) Storm surge maximum of the maximum (MOM). Accessed 28 Mar 2019
  53. NOAA (2017) Billion-dollar weather and climate disasters: table of events. NOAA National Centers for Environmental Information. Accessed 28 Mar 2019
  54. NOAA (2018) U.S. hit with 3 billion-dollar disasters in first 3 months of 2018. Accessed 28 Mar 2019
  55. NOAA National Centers for Environmental Information (NCEI) (2018) U.S. billion-dollar weather and climate disasters. Accessed 28 Mar 2019
  56. NOAA Storm Events (2019) Database Details. Accessed 28 Mar 2019
  57. Peduzzi P, Dao H, Herold C, Mouton F (2009) Assessing global exposure and vulnerability towards natural hazards: the Disaster Risk Index. Nat Hazards Earth Syst Sci 9(4):1149–1159CrossRefGoogle Scholar
  58. Perdikaris J, Gharabaghi B, McBean E (2011) A methodology for undertaking vulnerability assessments of flood susceptible communities. Int J Saf Secur Eng 1(2):126–146CrossRefGoogle Scholar
  59. Pielke RA Jr, Rubiera J, Landsea C, Fernández ML, Klein R (2003) Hurricane vulnerability in Latin America and the Caribbean: normalized damage and loss potentials. Nat Hazards Rev 4(3):101–114CrossRefGoogle Scholar
  60. Rothman M, Brown L (2007) The vulnerable geriatric casualty: medical needs of frail older adults during disasters. Generations 31(4):16–20Google Scholar
  61. Rygel L, O’sullivan D, Yarnal B (2006) A method for constructing a social vulnerability index: an application to hurricane storm surges in a developed country. Mitig Adapt Strat Glob Change 11(3):741–764CrossRefGoogle Scholar
  62. Saaty TL (1980) Analytic heirarchy process. Statistics reference online. Wiley, HobokenGoogle Scholar
  63. Schmidtlein MC, Deutsch RC, Piegorsch WW, Cutter SL (2008) A sensitivity analysis of the social vulnerability index. Risk Anal Int J 28(4):1099–1114CrossRefGoogle Scholar
  64. Schmidtlein MC, Shafer JM, Berry M, Cutter SL (2011) Modeled earthquake losses and social vulnerability in Charleston, South Carolina. Appl Geogr 31(1):269–281CrossRefGoogle Scholar
  65. Simpson DM, Human RJ (2008) Large-scale vulnerability assessments for natural hazards. Nat Hazards 47(2):143–155CrossRefGoogle Scholar
  66. Spielman SE, Folch D, Nagle N (2014) Patterns and causes of uncertainty in the American Community Survey. Appl Geogr 46:147–157CrossRefGoogle Scholar
  67. Tate E (2012) Social vulnerability indices: a comparative assessment using uncertainty and sensitivity analysis. Nat Hazards 63(2):325–347CrossRefGoogle Scholar
  68. Tate E (2013) Uncertainty analysis for a social vulnerability index. Ann As Am Geogr 103(3):526–543CrossRefGoogle Scholar
  69. R Core Team (2018) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
  70. Teknomo K (2017) Analytic hierarchy process (AHP). Accessed 28 Mar 2019
  71. Thacker MT, Lee R, Sabogal RI, Henderson A (2008) Overview of deaths associated with natural events, United States, 1979–2004. Disasters 32(2):303–315CrossRefGoogle Scholar
  72. Time (2017) ‘Need help asap’. The story behind the photo of nursing home residents trapped in hurricane flood water. U.S.—Texas. Accessed 28 Mar 2019
  73. Tobin GA, Montz BE (2004) Natural hazards and technology: vulnerability, risk, and community response in hazardous environments. In: Geography and technology. Springer, Dordrecht, pp 547–570CrossRefGoogle Scholar
  74. Wilson MJ, Sugg MM, Lane SJ (2019) Identifying multivariate vulnerability of nursing home facilities throughout the southeastern United States. Int J Disaster Risk Reduct 36:101–106CrossRefGoogle Scholar
  75. Wigtil G, Hammer RB, Kline JD, Mockrin MH, Stewart SI, Roper D, Radeloff VC (2016) Places where wildfire potential and social vulnerability coincide in the coterminous United States. Int J Wildland Fire 25(8):896–908CrossRefGoogle Scholar
  76. Zeileis A, Kleiber C, Jackman S (2008) Regression models for count data in R. J Stat Softw 27(8):1–25CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2020

Authors and Affiliations

  • Matthew Wilson
    • 1
  • Sandi Lane
    • 2
  • Raghuveer Mohan
    • 3
  • Margaret Sugg
    • 1
    Email author
  1. 1.Department of Geography and PlanningAppalachian State UniversityBooneUSA
  2. 2.Department of Health Care ManagementAppalachian State UniversityBooneUSA
  3. 3.Department of Computer ScienceAppalachian State UniversityBooneUSA

Personalised recommendations