Advertisement

Natural Hazards

, Volume 37, Issue 3, pp 315–329 | Cite as

Validation and Evaluation of Predictive Models in Hazard Assessment and Risk Management

  • Santiago Beguería
Article

Abstract

The paper deals with the validation and evaluation of mathematical models in natural hazard analysis, with a special focus on establishing their predictive power. Although most of the tools and statistics available are common to general classification models, some peculiarities arise in the case of hazard assessment. This is due to the fact that the target for validation, the propensity to develop a dangerous characteristic, is not really known and must be estimated from a (usually) very small sample. This implies that the two types of errors (false positives and false negatives) should be given different meanings. Related to this, a very frequent situation is the presence of prevalence (different proportion of positive and negative cases) in the sample. It is shown that sample prevalence can have a dramatic effect in some very common validation statistics, like the confusion matrix and model efficiency. Here some statistics based on the confusion matrix are presented and discussed, and the use of threshold-independent approaches (especially the ROC plot) is shown. The ROC plot is also proposed as a convenient tool for decision-taking in a risk management context. A general scheme for hazard predictive modeling is finally proposed.

Keywords

geomorphological hazard modelling probabilistic models prediction errors accuracy assessment decision support decision threshold ROC plot 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bledsoe, B. P., Watson, C. C. 2001Logistic analysis of channel pattern thresholds: meandering, braiding, and incisingGeomorphology38281300Google Scholar
  2. Carrara, A. 1983Multivariate models for landslide hazard evaluationMath. Geol.15403426Google Scholar
  3. Carrara, A., Cardinali, M., Detti, R., Guzzetti, F., Pasqui, V., Reichenbach, P. 1991GIS techniques and statistical models in evaluating landslide hazardEarth Surf. Process. Landforms16427445Google Scholar
  4. Chung, C. F., Fabbri, A. G. 1999Probabilistic prediction models for landslide hazard mappingPhotogram. Eng. Remote Sensing6513891399Google Scholar
  5. Chung, C. F., Fabbri, A. G. 2003Validation of spatial prediction models for landslide hazard mappingNat. Hazards30451472CrossRefGoogle Scholar
  6. Chung, C. F., Fabbri, A., and Van Westen, C. J.: 1995. Multivariate regression analysis for landslide hazard zonation. In A. Carrara & F. Guzzetti (Eds.), The Nederlands: Kluwer Academic Publishers (pp. 107–133).Google Scholar
  7. Clerici, A., Parego, S., Tellini, C., Vescovi, P. 2002A procedure for landslide susceptibility zonation by the conditional analysis methodGeomorphology48349364CrossRefGoogle Scholar
  8. Dai, F. C., Lee, C. F. 2002Landslide characteristics and slope instability modeling using GIS, Lantau Island, Hong KongGeomorphology42213228Google Scholar
  9. Deleo, J. M.: 1993, Receiver operating characteristic laboratory (ROCLAB): software for developing decision strategies that account for uncertainty. In: Proceedings of the Second International Symposium on Uncertainty Modelling and Analysis, Computer Society Press, College Park, pp. 318–325Google Scholar
  10. Fielding, A. H., Bell, J. F. 1997A review of methods for the assessment of prediction errors in conservation presence/absence modelsEnviron. Conserv.243849Google Scholar
  11. Floyer, J. A., McClung, D. M. 2003Numerical avalanche prediction: Bear Pass, British Columbia, CanadaCold Regions Sci. Technol.37333342Google Scholar
  12. Forbes, A. D. 1995Classification-algorithm evaluation: five performance measures based on confusion matricesJ. Clin. Monitor.11189206Google Scholar
  13. Furbish, D. J., Rice, R. M. 1983Predicting landslides related to clearcut logging, Northwestern California, USAMount. Res. Dev.3253259Google Scholar
  14. Lee, S., Choi, J., Min, K. 2002Landslide susceptibility analysis and verification using the Bayesian probability modelEnviron. Geol.43120131Google Scholar
  15. Lorente, A., García-Ruiz, J. M., Beguería, S., Arnáez, J. 2002Factors explaining the spatial distribution of hillslope debris flows: a case study in the Flysch Sector of the Central Spanish PyreneesMount. Res. Dev.223239Google Scholar
  16. Martínez-Casasnovas, J. A., Ramos, M. C., Poesen, J. 2003Assessment of sidewall erosion in large gullies using multi-temporal DEMs and logistic regression analysisGeomorphology58305321Google Scholar
  17. Massie, D. D., White, K. D., Daly, S. F. 2002Application of neural networks to predict ice jam occurrenceCold Regions Sci. Technol.35115122Google Scholar
  18. Morgan, R. P. C., Mngomezulu, D. 2003Threshold conditions for initiation of valley-side gullies in the Middle Veld of SwazilandCatena50401414Google Scholar
  19. Neuland, H. 1976A prediction model of landslipsCatena3215230CrossRefGoogle Scholar
  20. Perry, F. V., Valentine, G. A., Desmarais, E. K., WoldeGabriel, G. 2001Probabilistic assessment of volcanic hazard to radioactive waste repositories in Japan: intersection by a dike from a nearby composite volcanoGeology29255258CrossRefGoogle Scholar
  21. Remondo, J., González-Díez, A., Terán, J. R. D. D., Cendrero, A. 2003Landslide susceptibility models utilising spatial data analysis techniques. A case study from the Lower Deba Valley, Guipuzcoa (Spain)Nat. Hazards30233249Google Scholar
  22. Rice, R. M. and Pillsbury, N. H.: 1982, Predicting landslides in clearcut patches. Symposium on Recent Development in the Explanation and Prediction of Erosion and Sediment Yield. International Association of Hydrological Sciences, pp. 303–311Google Scholar
  23. Rowbotham, D. N., Dudycha, D. 1998GIS modelling of slope stability in Phewa Tal watershed, NepalGeomorphology26151170Google Scholar
  24. Beek, R., Asch, T. 2004Regional assessment of the effects of land-use change on landslide hazard by means of physically based modellingNat. Hazards31289304Google Scholar
  25. Westen, C. J., Rengers, N., Terlien, M. T. J., Soeters, R. 1997Prediction of the occurrence of slope instability phenomenal through GIS-based hazard zonationGeol. Runds. (Int. J. Earth Sci.)86404414Google Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  1. 1.Division Landscape Dynamics, GIS and Hydrology – Faculty of GeosciencesUtrecht UniversityUtrechtThe Netherlands

Personalised recommendations