Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Estimating landslide susceptibility areas considering the uncertainty inherent in modeling methods


Landslides are one of the most dangerous types of natural disasters, and damage due to landslides has been increasing in certain regions of the world because of increased precipitation. Policy decision makers require reliable information that can be used to establish spatial adaptation plans to protect people from landslide hazards. Researchers presently identify areas susceptible to landslides using various spatial distribution models. However, such data are associated with a high amount of uncertainty. This study focuses on quantifying the uncertainty of several spatial distribution models and identifying the effectiveness of various ensemble methods that can be used to provide reliable information to support policy decisions. The area of study was Inje-gun, Republic of Korea. Ten models were selected to assess landslide susceptibility. Moreover, five ensemble methods were selected for the aggregated results of the 10 models. The uncertainty was quantified using the coefficient of variation and the uncertainty map we developed revealed areas with strongly differing values among single models. A matrix map was created using an ensemble map and a coefficient of variation map. Using matrix analysis, we identified the areas that are most susceptible to landslides according to the ensemble model with a low uncertainty. Thus, the ensemble model can be a useful tool for supporting decision makers. The framework of this study can also be employed to support the establishment of landslide adaptation plans in other areas of the Republic of Korea and in other countries.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    An ROC curve is a plot graph that shows the diagnostic ability of a binary classification method considering a threshold value. ROC curves are created by plotting the true positive rate (TPR) against the false positive rate (FPR). The TPR is also called the sensitivity and the FPR is also known as the probability of false alarm, and it can be calculated as (1 − specificity). Thus, the ROC curve represents the sensitivity as a function of FPR. ROC analysis is used as a tool to select optimal models and to discard suboptimal ones. The area under the curve (AUC) is the same as the probability that a classifier will grade a randomly chosen positive case higher than a randomly chosen negative case. The AUC is similar to the Mann–Whitney U, which tests whether positive cases are graded higher than negative cases.

  2. 2.

    Coefficient of variation is also known as relative standard deviation. It is a standardized value of dispersion of a probability distribution. It is calculated by the ratio of the standard deviation to the mean. In this study, it is used for quantifying uncertainty of modeling results.


  1. Akgun A (2012) A comparison of landslide susceptibility maps produced by logistic regression, multi-criteria decision, and likelihood ratio methods: a case study at Izmir, Turkey. Landslides 9:93–106

  2. Akgun A, Dag S, Bulut F (2008) Landslide susceptibility mapping for a landslide-prone area (Findikli, NE of Turkey) by likelihood-frequency ratio and weighted linear combination models. Environ Geol 54:1127–1143

  3. Althuwaynee OF, Pradhan B, Park H-J, Lee JH (2014) A novel ensemble bivariate statistical evidential belief function with knowledge-based analytical hierarchy process and multivariate statistical logistic regression for landslide susceptibility mapping. CATENA 114:21–36

  4. Ayalew L, Yamagishi H (2005) The application of GIS-based logistic regression for landslide susceptibility mapping in the Kakuda-Yahiko Mountains, Central Japan. Geomorphology 65:15–31

  5. Bartlett P, Shawe-Taylor J (1999) Generalization performance of support vector machines and other pattern classifiers. In: Advances in Kernel Methods—Support Vector Learning, pp 43–54

  6. Bonachea J, Remondo J, Terán D et al (2009) Landslide risk models for decision making. Risk Anal 29:1629–1643

  7. Boo K, Kwon W, Baek H (2006) Change of extreme events of temperature and precipitation over Korea using regional projection of future climate change. Geophys Res Lett 33:8–11.

  8. Bühlmann P, Yu B (2003) Boosting with the L 2 loss: regression and classification. J Am Stat Assoc 98:324–339

  9. Bui DT, Pradhan B, Lofman O et al (2012) Landslide susceptibility mapping at Hoa Binh province (Vietnam) using an adaptive neuro-fuzzy inference system and GIS. Comput Geosci 45:199–211

  10. Buisson L, Thuiller W, Casajus N et al (2010) Uncertainty in ensemble forecasting of species distribution. Glob Change Biol 16:1145–1157

  11. Catani F, Casagli N, Ermini L et al (2005) Landslide hazard and risk mapping at catchment scale in the Arno River basin. Landslides 2:329–342

  12. Chiou I-J, Chen C-H, Liu W-L et al (2015) Methodology of disaster risk assessment for debris flows in a river basin. Stoch Environ Res Risk Assess 29:775–792.

  13. Choi JH, Oh JY, Kim YS, Kim HT (2011) Analysis of the controlling factors of an urban-type landslide at Hwangryeong mountain based on tree growth patterns and geomorphology. J Eng Geol 21:281–293

  14. Claessens L, Schoorl JM, Veldkamp A (2007) Modelling the location of shallow landslides and their effects on landscape dynamics in large watersheds: an application for Northern New Zealand. Geomorphology 87:16–27

  15. Clinton N, Yu L, Gong P (2015) Geographic stacking: decision fusion to increase global land cover map accuracy. ISPRS J Photogramm Remote Sens 103:57–65.

  16. Dehnavi A, Aghdam IN, Pradhan B, Varzandeh MHM (2015) A new hybrid model using step-wise weight assessment ratio analysis (SWARA) technique and adaptive neuro-fuzzy inference system (ANFIS) for regional landslide hazard assessment in Iran. CATENA 135:122–148

  17. Dimitriadou E, Weingessel A, Hornik K (2003) A cluster ensembles framework. IOS Press, Amsterdam

  18. Ermini L, Catani F, Casagli N (2005) Artificial neural networks applied to landslide susceptibility assessment. Geomorphology 66:327–343

  19. Felicísimo ÁM, Cuartero A, Remondo J, Quirós E (2013) Mapping landslide susceptibility with logistic regression, multiple adaptive regression splines, classification and regression trees, and maximum entropy methods: a comparative study. Landslides 10:175–189

  20. Franklin J (2009) Mapping species distributions. Cambridge University Press, Cambridge

  21. Ghosh J, Acharya A (2011) Cluster ensembles. Wiley Interdiscip Rev Data Min Knowl Discov 1:305–315

  22. Guzzetti F, Peruccacci S, Rossi M, Stark CP (2008) The rainfall intensity–duration control of shallow landslides and debris flows: an update. Landslides 5:3–17

  23. Hansson SL, Röjvall AS, Rastam M et al (2005) Psychiatric telephone interview with parents for screening of childhood autism–tics, attention-deficit hyperactivity disorder and other comorbidities (A–TAC) Preliminary reliability and validity. Br J Psychiatry 187:262–267

  24. Kappes MS, Papathoma-Köhle M, Keiler M (2012) Assessing physical vulnerability for multi-hazards using an indicator-based methodology. Appl Geogr 32:577–590

  25. Kim J (2012) The analysis of planning methode and case study for Model “Climate Change Adaptation City”. J Korea Inst Ecol Archit Environ 12:13–19

  26. Kim WY, Chae BG (2009) Characteristics of rainfall, geology and failure geometry of the landslide areas on natural terrains, Korea. J Eng Geol 19:331–344

  27. Kim GH, Yune CY, Lee HG, Hwang JS (2011a) Debris flow analysis of landslide area in Inje using GIS. J Korean Soc Surv Geod Photogramm Cartogr 29:47–53

  28. Kim KH, Jung HR, Park JH, Ma HS (2011b) Analysis on rainfall and geographical characteristics of landslides in Gyeongnam Province. J Korean Environ Restor Technol 14:33–45

  29. Kim HG, Lee DK, Park C et al (2015) Evaluating landslide hazards using RCP 4.5 and 8.5 scenarios. Environ Earth Sci 73:1385–1400

  30. Korea Ministry of Environment (KME) (2005) GIS map of forest type for Republic of Korea

  31. Korea Ministry of Environment (KME) (2008) Digital elevation model for Republic of Korea

  32. Korea Meteorological Administration (2011) White Paper for Typhoon of Republic of Korea

  33. Kovalchuk SV, Krikunov AV, Knyazkov KV, Boukhanovsky AV (2017) Classification issues within ensemble-based simulation: application to surge floods forecasting. Stoch Environ Res Risk Assess 31:1183–1197.

  34. Krishnamurti TN, Kishtawal CM, Zhang Z et al (2000) Multimodel ensemble forecasts for weather and seasonal climate. J Clim 13:4196–4216

  35. Kwon HS (2014) Applying ensemble model for identifying uncertainty in the species distribution models. J Korean Soc Geospat Inf Syst 2955:47–52

  36. Ladle R, Hortal J (2013) Mapping species distributions: living with uncertainty. Front Biogeogr 5:4–6

  37. Lee S, Oh H-J (2012) Ensemble-based landslide susceptibility maps in Jinbu area, Korea. In: Pradhan B, Buchroithner M (eds) Terrigenous mass movements. Springer, Berlin, pp 193–220

  38. Lee M-J, Park I, Lee S (2015) Forecasting and validation of landslide susceptibility using an integration of frequency ratio and neuro-fuzzy models: a case study of Seorak mountain area in Korea. Environ Earth Sci 74:413–429

  39. Li Y, Chen G, Tang C et al (2012) Rainfall and earthquake-induced landslide susceptibility assessment using GIS and artificial neural network. Nat Hazards Earth Syst Sci 12:2719–2729

  40. Lian C, Zeng Z, Yao W, Tang H (2014) Extreme learning machine for the displacement prediction of landslide under rainfall and reservoir level. Stoch Environ Res Risk Assess 28:1957–1972.

  41. Liu K, Yao C, Chen J et al (2016) Comparison of three updating models for real time forecasting: a case study of flood forecasting at the middle reaches of the Huai River in East China. Stoch Environ Res Risk Assess 31:1–14.

  42. Miao F, Wu Y, Xie Y et al (2016) Research on progressive failure process of Baishuihe landslide based on Monte Carlo model. Stoch Environ Res Risk Assess.

  43. Niedzielski T, Miziński B (2016) Real-time hydrograph modelling in the upper Nysa Klodzka river basin (SW Poland): a two-model hydrologic ensemble prediction approach. Stoch Environ Res Risk Assess.

  44. Nourani V, Pradhan B, Ghaffari H, Sharifi SS (2014) Landslide susceptibility mapping at Zonouz Plain, Iran using genetic programming and comparison with frequency ratio, logistic regression, and artificial neural network models. Nat Hazards 71:523–547

  45. Oh HJ (2010) Landslide detection and landslide susceptibility mapping using aerial photos and artificial neural networks. Korean J Remote Sens 26:47–57

  46. Oh CY, Choi CU, Kim KT (2009) Analysis of landslide characteristics of Inje area using SPOT5 images and GIS analysis. Korean J Remote Sens 25:445–454

  47. Park S, Choi C, Kim B, Kim J (2013) Landslide susceptibility mapping using frequency ratio, analytic hierarchy process, logistic regression, and artificial neural network methods at the Inje area, Korea. Environ Earth Sci 68:1443–1464

  48. Pradhan B (2013) A comparative study on the predictive ability of the decision tree, support vector machine and neuro-fuzzy models in landslide susceptibility mapping using GIS. Comput Geosci 51:350–365

  49. Pradhan B, Lee S (2010) Delineation of landslide hazard areas on Penang Island, Malaysia, by using frequency ratio, logistic regression, and artificial neural network models. Environ Earth Sci 60:1037–1054

  50. Riebau AR, Fox DG (2005) Damage assessment of agrometeorological relevance from natural disasters: economic and social consequences. In: Sivakumar MV, Motha RP, Das HP (eds) Natural disasters and extreme events in agriculture. Springer, Berlin, pp 119–135

  51. Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33:1–39

  52. Son JW, Kim KT, Lee CH, Choi CU (2009) Analysis of landslide in Inje region using aerial photograph and GIS. J Korean Soc Geo-spat Inf Syst 17:61–69.

  53. Son CY, Kim JS, Il Moon Y, Lee JH (2014) Characteristics of tropical cyclone-induced precipitation over the Korean River basins according to three evolution patterns of the Central-Pacific El Nino. Stoch Environ Res Risk Assess 28:1147–1156.

  54. Sudmeier-Rieux K, Jaquet S, Derron M-H et al (2012) A case study of coping strategies and landslides in two villages of Central-Eastern Nepal. Appl Geogr 32:680–690

  55. Sung JH, Kang H-S, Park S et al (2012) Projection of extreme precipitation at the end of 21st Century over South Korea based on Representative Concentration Pathways (RCP). Korean Meteorol Soc 22:221–231

  56. Thuiller W, Lafourcade B, Engler R, Araújo MB (2009) BIOMOD—a platform for ensemble forecasting of species distributions. Ecography (Cop) 32:369–373

  57. Thuiller W, Georges D, Engler R (2015) biomod2 package manual

  58. Tongal H, Booij MJ (2017) Quantification of parametric uncertainty of ANN models with GLUE method for different streamflow dynamics. Stoch Environ Res Risk Assess 31:993–1010.

  59. Torizin J (2016) Elimination of informational redundancy in the weight of evidence method: an application to landslide susceptibility assessment. Stoch Environ Res Risk Assess 30:635–651.

  60. Wang X, Yang T, Li X et al (2016a) Spatio-temporal changes of precipitation and temperature over the Pearl River basin based on CMIP5 multi-model ensemble. Stoch Environ Res Risk Assess 31:1–13.

  61. Wang YY, Huang GH, Wang S (2016b) CVaR-based factorial stochastic optimization of water resources systems with correlated uncertainties. Stoch Environ Res Risk Assess 31:1543–1553.

  62. Water Resources Management Information System (WAMIS) (2006) GIS map of soil for Republic of Korea

  63. Yeon YK (2011) Evaluation and analysis of Gwangwon-do landslide susceptibility using logistic regression. J Korean Assoc Geogr Inf Stud 14:116–127

  64. Yesilnacar E, Topal T (2005) Landslide susceptibility mapping: a comparison of logistic regression and neural networks methods in a medium scale study, Hendek region (Turkey). Eng Geol 79:251–266

  65. Yilmaz I (2009) Landslide susceptibility mapping using frequency ratio, logistic regression, artificial neural networks and their comparison: a case study from Kat landslides (Tokat—Turkey). Comput Geosci 35:1125–1138

  66. Yilmaz I (2010) Comparison of landslide susceptibility mapping methodologies for Koyulhisar, Turkey: conditional probability, logistic regression, artificial neural networks, and support vector machine. Environ Earth Sci 61:821–836

  67. Yoo N, Yoon B, Um D et al (2012) Analysis of rainfall characteristics and landslides at the west side area of Gangwon Province. J Korean Geoenviron Soc 13:75–82

  68. Zare M, Pourghasemi HR, Vafakhah M, Pradhan B (2013) Landslide susceptibility mapping at Vaz Watershed (Iran) using an artificial neural network model: a comparison between multilayer perceptron (MLP) and radial basic function (RBF) algorithms. Arab J Geosci 6:2873–2888

Download references


This work was supported by Korea Ministry of Environment (MOE, Project No. 2016000210004) as “Public Technology Program based on Environmental Policy” and the BK 21 Plus Project in 2015 (Seoul National University Interdisciplinary Program in Landscape Architecture, Global Leadership Program toward Innovative Green Infrastructure).

Author information

Correspondence to Dong Kun Lee.


Appendix 1: Key features of the 10 SDMs (Franklin 2009)

Model Full name of model Category Occurrence data required Response function Features of the model
GLM Generalized linear model Statistically based model Occurrence/No occurrence Parametric linear, polynomial, piecewise, interaction terms GLMs are a representative model among SDMs. GLMs are a generalization of the multiple regression model that uses the link function to accommodate response variables that are distributed normally, namely, the response distributions
GAM Generalized additive model Statistically based model Occurrence/No occurrence Smoothing function, estimated using local regression, splines or other methods GAMs in SDMs are suggested as a powerful methodology to detect and describe non-linear response functions. The results of GAMs can be used to build a parametric model
MARS Multivariate adaptive regression splines Statistically based model Occurrence/No occurrence Adaptive piecewise linear regression combines splines and binary recursive partitioning MARS can give a type of a generalization of a stepwise linear regression. MARS are suited to analyses with large numbers of variables or a modification of the regression tree method
GBM Generalized boosted regression model Machine learning based model Occurrence/No occurrence Weighted and unweighted model averaging applied to decision trees GBMs are similar to weighting variables that consider higher probabilities of selection, instead of weighting equal probabilities for subsequent variables
CTA Classification tree analysis Machine learning based model Occurrence/No occurrence Divisive, monothetic decision rules (thresholds) from binary recursive partitioning The goal of CTA is to divide data into homogeneous subgroups. The subgroups consist of variables that have similar values or are in the same class in regard to the ranges of values for the variables
ANN Artificial neural network Machine learning based model Occurrence/No occurrence Non-linear decision boundaries in covariate space ANN can be described as a two-stage classification or regression model. A hidden layer of ANN comprises features that are linear combinations of input variables. The output variable is a weighted combination of features in the hidden layer
SRE Rectilinear envelope similar to BIOCLIM Machine learning based model Occurrence only Fuzzy classification approach SRE is a boxcar or parallelepiped classifier that uses BIOCLIM. SRE assesses the potential distribution of the dependent variable by using the multi-dimensional environmental space bounded by the values for all dependent variables
MDA Mixture discriminant analysis Machine learning based model Occurrence/No occurrence Linear MDA is a type of linear discriminant analysis that models the multivariate density of variables by using a mixture of multivariate normal distributions
RF Random forest Machine learning based model Occurrence/No occurrence Weighted and unweighted model averaging applied to decision trees Random forests is a type of bootstrap aggregating method that builds de-correlated trees and averages the trees. Many trees are constructed with subsets of input data. Furthermore, each division of the tree model is also constructed with a random subset of input variables
MAXENT Maximum entropy algorithm Machine learning based model Occurrence only Non-linear response functions can be described Maximum entropy is based on statistical mechanics and information theory. MAXENT can analyze the best approximation of an unknown distribution by using the maximum entropy method, which considers the most spread out and closest to uniform values

Appendix 2: Maps of variables for landslide susceptibility model


Appendix 3: Landslide projections of the 10 models for present conditions



2. CTA


3. SRE


4. FDA




6. RF


7. GLM


8. GBM


9. GAM


10. ANN


Appendix 4: ROC plots for each model

Axis Value Scope
X Value of each variable The scope of value varies on variables. Please see the Table 1
Y Probability of landslide occurrence 0–1



2. CTA


3. SRE


4. FDA




6. RF


7. GLM


8. GBM


9. GAM


10. ANN


Appendix 5: Results of ensemble models for the present conditions

1. PM


2. PCI lower


3. PCI upper


4. PME


5. CA


6. PMW


Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, H.G., Lee, D.K., Park, C. et al. Estimating landslide susceptibility areas considering the uncertainty inherent in modeling methods. Stoch Environ Res Risk Assess 32, 2987–3019 (2018).

Download citation


  • Spatial distribution model
  • Ensemble model
  • Coefficient of variation
  • Adaptation plans