Skip to main content

Advertisement

SpringerLink
Log in

Model performance analysis for landslide susceptibility in cold regions using accuracy rate and fluctuation characteristics

  • Original Paper
  • Published:
Natural Hazards Aims and scope Submit manuscript

Abstract

Considering the increasing number of landslides due to permafrost degradation, this paper reports a performance evaluation of three classical landslide susceptibility models applied to cold regions. A landslide inventory was first constructed through a historical survey and image interpretation. Ten causative factors of landslides were then chosen based on the available data and the local environment. Multicollinearity diagnosis and factor effectiveness test were employed to perform a factor analysis. Subsequently, three evaluation models based on the frequency ratio (FR), logistic regression (LR), and artificial neural network (ANN) were established. These models were applied to obtain landslide susceptibility maps, which were then analyzed and compared. The model performance was evaluated in terms of the accuracy rate and fluctuation characteristics. The results showed no multicollinearity issue between the factors employed. The annual temperature difference and frozen depth are two indispensable factors when assessing landslide susceptibility in cold regions. A comparison between the susceptibility maps generated using the three models showed that the FR model-generated map is most in line with the principle of disaster zoning and has the highest degree of conformity with actual landslide points, followed by the maps generated using the LR and ANN models. An accuracy analysis showed that the ANN model yields the highest AUC value in the training and test states, 0.957 and 0.863, respectively; however, these values were not optimal given the fluctuation. Moreover, the fluctuation in the non-landslide data was greater than that in the landslide data. The fluctuation results revealed the drawback of the AUC value in the analysis of the model performance. In other words, the non-landslide error often covers up the landslide error. This study provides a scientific guidance for evaluating the model performance and for assessing landslide disasters in cold regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Al-Abadi AM, Al-Najar NA (2019) Comparative assessment of bivariate, multivariate and machine learning models for mapping flood proneness. Nat Hazards 100:461–491

    Article  Google Scholar 

  • Allen S, Huggel C (2013) Extremely warm temperatures as a potential cause of recent high mountain rockfall. Global Planet Change 107:59–69

    Article  Google Scholar 

  • Allen SK, Cox SC, Owens IF (2010) Rock avalanches and other landslides in the central Southern Alps of New Zealand: a regional study considering possible climate change impacts. Landslides 8:33–48

    Article  Google Scholar 

  • Barella CF, Sobreira FG, Zêzere JL (2018) A comparative analysis of statistical landslide susceptibility mapping in the southeast region of Minas Gerais state, Brazil. Bull Eng Geol Env 78:3205–3221

    Article  Google Scholar 

  • Blikra LH, Christiansen HH (2014) A field-based model of permafrost-controlled rockslide deformation in northern Norway. Geomorphology 208:34–49

    Article  Google Scholar 

  • Cantarino I, Carrion MA, Goerlich F, Martinez Ibañez V (2018) A ROC analysis-based classification method for landslide susceptibility maps. Landslides 16:265–282

    Article  Google Scholar 

  • Chae B-G, Park H-J, Catani F, Simoni A, Berti M (2017) Landslide prediction, monitoring and early warning: a concise review of state-of-the-art. Geosci J 21:1033–1070

    Article  Google Scholar 

  • Chen W, Shirzadi A, Shahabi H, Ahmad BB, Zhang S, Hong H, Zhang N (2017) A novel hybrid artificial intelligence approach based on the rotation forest ensemble and naïve Bayes tree classifiers for a landslide susceptibility assessment in Langao County, China. Geomat Nat Haz Risk 8:1955–1977

    Article  Google Scholar 

  • Chen W, Yan X, Zhao Z, Hong H, Bui DT, Pradhan B (2018) Spatial prediction of landslide susceptibility using data mining-based kernel logistic regression, naive Bayes and RBFNetwork models for the Long County area (China). Bull Eng Geol Environ 78(1):247–266

    Article  Google Scholar 

  • Delan X, Changlin G (2011) The characteristics of thegeological environment and geological disasters, Heilongjiang Province. Chin J Geol Hazard Control 22:101–106

    Google Scholar 

  • Ding Y, Zhang S, Zhao L, Li Z, Kang S (2019) Global warming weakening the inherent stability of glaciers and permafrost. Sci Bull 64:245–253

    Article  Google Scholar 

  • Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, Marquéz JRG, Gruber B, Lafourcade B, Leitão PJ, Münkemüller T, McClean C, Osborne PE, Reineking B, Schröder B, Skidmore AK, Zurell D, Lautenbach S (2013) Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 36:27–46

    Article  Google Scholar 

  • Dou J, Yunus AP, Merghadi A, Shirzadi A, Nguyen H, Hussain Y, Avtar R, Chen Y, Pham BT, Yamagishi H (2020) Different sampling strategies for predicting landslide susceptibilities are deemed less consequential with deep learning. Sci Total Environ 720:137320

    Article  Google Scholar 

  • Fattahi H, Zandy Ilghani N (2020) Slope Stability Analysis Using Bayesian Markov Chain Monte Carlo Method. Geotech Geol Eng 38:2609–2618

    Article  Google Scholar 

  • Fell R, Corominas J, Bonnard C, Cascini L, Leroi E, Savage WZ (2008) Guidelines for landslide susceptibility, hazard and risk zoning for land-use planning. Eng Geol 102:99–111

    Article  Google Scholar 

  • Feng J, Li N, Zhang Z, Chen X (2017) How to apply the dependence structure analysis to extreme temperature and precipitation for disaster risk assessment. Theoret Appl Climatol 133:297–305

    Article  Google Scholar 

  • Gao J, Liu Y (2011) Climate warming and land use change in Heilongjiang Province, Northeast China. Appl Geogr 31:476–482

    Article  Google Scholar 

  • Guzzetti F, Reichenbach P, Cardinali M, Galli M, Ardizzone F (2005) Probabilistic landslide hazard assessment at the basin scale. Geomorphology 72:272–299

    Article  Google Scholar 

  • Haeberli W (2013) Mountain permafrost—research frontiers and a special long-term challenge. Cold Reg Sci Technol 96:71–76

    Article  Google Scholar 

  • Hong H, Miao Y, Liu J, Zhu AX (2019) Exploring the effects of the design and quantity of absence data on the performance of random forest-based landslide susceptibility mapping. CATENA 176:45–64

    Article  Google Scholar 

  • Huang Y, Zhao L (2018) Review on landslide susceptibility mapping using support vector machines. CATENA 165:520–529

    Article  Google Scholar 

  • Huggel C, Salzmann N, Allen S, Caplan-Auerbach J, Fischer L, Haeberli W, Larsen C, Schneider D, Wessels R (2010) Recent and future warm extreme events and high-mountain slope stability. Philos Trans A Math Phys Eng Sci 368:2435–2459

    Google Scholar 

  • Kia MB, Pirasteh S, Pradhan B, Mahmud AR, Sulaiman WNA, Moradi A (2011) An artificial neural network model for flood simulation using GIS: Johor River Basin, Malaysia. Environ Earth Sci 67:251–264

    Article  Google Scholar 

  • Lan H, Derek Martin C, Lim CH (2007) RockFall analyst: A GIS extension for three-dimensional and spatially distributed rockfall hazard modeling. Comput Geosci 33:262–279

    Article  Google Scholar 

  • Lan H, Martin CD, Zhou C, Lim CH (2010) Rockfall hazard analysis using LiDAR and spatial modeling. Geomorphology 118:213–223

    Article  Google Scholar 

  • Lee YH, Kim JR, Kim D, Kang HB (2007) New Approach for Estimating Rock Slope Stability Using Logistic Regression Analysis. Transp Res Rec J Transp Res Board 2016:99–109

    Article  Google Scholar 

  • Li L, Lan H (2015) Probabilistic modeling of rockfall trajectories: a review. Bull Eng Geol Env 74:1163–1176

    Article  Google Scholar 

  • Li C, Ma T, Sun L, Li W, Zheng A (2011) Application and verification of a fractal approach to landslide susceptibility mapping. Nat Hazards 61:169–185

    Article  Google Scholar 

  • Li L, Lan H, Guo C, Zhang Y, Li Q, Wu Y (2016) A modified frequency ratio method for landslide susceptibility assessment. Landslides 14:727–741

    Article  Google Scholar 

  • Luo X, Lin F, Chen Y, Zhu S, Xu Z, Huo Z, Yu M, Peng J (2019) Coupling logistic model tree and random subspace to predict the landslide susceptibility areas with considering the uncertainty of environmental features. Sci Rep 9:15369

    Article  Google Scholar 

  • Palenzuela Baena JA, Scifoni S, Marsella M, De Astis G, Irigaray Fernández C (2019) Landslide susceptibility mapping on the islands of Vulcano and Lipari (Aeolian Archipelago, Italy), using a multi-classification approach on conditioning factors and a modified GIS matrix method for areas lacking in a landslide inventory. Landslides 16:969–982

    Article  Google Scholar 

  • Patton AI, Rathburn SL, Capps DM (2019) Landslide response to climate change in permafrost regions. Geomorphology 340:116–128

    Article  Google Scholar 

  • Pham VD, Nguyen Q-H, Nguyen H-D, Pham V-M, Vu VM, Bui Q-T (2020) Convolutional Neural Network—Optimized Moth Flame Algorithm for Shallow Landslide Susceptible Analysis. IEEE Access 8:32727–32736

    Article  Google Scholar 

  • Pourghasemi HR, Kornejady A, Kerle N, Shabani F (2020) Investigating the effects of different landslide positioning techniques, landslide partitioning approaches, and presence-absence balances on landslide susceptibility mapping. CATENA 187:104364

    Article  Google Scholar 

  • Pradhan AMS, Kim Y-T (2016) Spatial data analysis and application of evidential belief functions to shallow landslide susceptibility mapping at Mt. Umyeon, Seoul, Korea. Bull Eng Geol Env 76:1263–1279

    Article  Google Scholar 

  • Pradhan B, Lee S (2009) Regional landslide susceptibility analysis using back-propagation neural network model at Cameron Highland, Malaysia. Landslides 7:13–30

    Article  Google Scholar 

  • Pradhan B, Lee S (2010) Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modelling. Environ Model Softw 25:747–759

    Article  Google Scholar 

  • Reichenbach P, Rossi M, Malamud BD, Mihir M, Guzzetti F (2018) A review of statistically-based landslide susceptibility models. Earth Sci Rev 180:60–91

    Article  Google Scholar 

  • Sameen MI, Pradhan B, Bui DT, Alamri AM (2020) Systematic sample subdividing strategy for training landslide susceptibility models. CATENA 187:104358

    Article  Google Scholar 

  • Schaub Y, Haeberli W, Huggel C, Künzler M, Bründl M (2013) Landslides and New Lakes in Deglaciating Areas: A Risk Management Framework. Springer, Berlin

    Google Scholar 

  • Shafapour Tehrany M, Shabani F, Neamah Jebur M, Hong H, Chen W, Xie X (2017) GIS-based spatial prediction of flood prone areas using standalone frequency ratio, logistic regression, weight of evidence and their ensemble techniques. Geomat Nat Haz Risk 8:1538–1561

    Article  Google Scholar 

  • Shao X, Ma S, Xu C, Zhou Q (2020) Effects of sampling intensity and non-slide/slide sample ratio on the occurrence probability of coseismic landslides. Geomorphology 363:107222

    Article  Google Scholar 

  • Sharma S, Mahajan AK (2019) Information value based landslide susceptibility zonation of Dharamshala region, northwestern Himalaya, India. Spat Inf Res 27:553–564

    Article  Google Scholar 

  • Shirzadi A, Saro L, Hyun Joo O, Chapi K (2012) A GIS-based logistic regression model in rock-fall susceptibility mapping along a mountainous road: Salavat Abad case study, Kurdistan. Iran Nat Hazards 64:1639–1656

    Article  Google Scholar 

  • Soma AS, Kubota T, Mizuno H (2019) Optimization of causative factors using logistic regression and artificial neural network models for landslide susceptibility assessment in Ujung Loe Watershed, South Sulawesi Indonesia. J Mt Sci 16:383–401

    Article  Google Scholar 

  • Steger S, Brenning A, Bell R, Petschko H, Glade T (2016) Exploring discrepancies between quantitative validation results and the geomorphic plausibility of statistical landslide susceptibility maps. Geomorphology 262:8–23

    Article  Google Scholar 

  • Tehrany MS, Shabani F, Jebur MN, Hong H, Chen W, Xie X (2017) GIS-based spatial prediction of flood prone areas using standalone frequency ratio, logistic regression, weight of evidence and their ensemble techniques. Geom Nat Hazards Risk 8(2):1538–1561

    Article  Google Scholar 

  • Thai Pham B, Shirzadi A, Tien Bui D, Prakash I, Dholakia MB (2018) A hybrid machine learning ensemble approach based on a Radial Basis Function neural network and Rotation Forest for landslide susceptibility modeling: a case study in the Himalayan area, India. Int J Sediment Res 33(2):157–170

    Article  Google Scholar 

  • Thi Ngo PT, Panahi M, Khosravi K, Ghorbanzadeh O, Kariminejad N, Cerda A, Lee S (2021) Evaluation of deep learning algorithms for national scale landslide susceptibility mapping of Iran. Geosci Front 12:505–519

    Article  Google Scholar 

  • Tien Bui D, Tuan TA, Klempe H, Pradhan B, Revhaug I (2016) Spatial prediction models for shallow landslide hazards: a comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides 13:361–378

    Article  Google Scholar 

  • Wenmei Z (2015) Study on physical and mechanical properties of permafrost at high altitudes. Forest Eng 31:128–131

    Google Scholar 

  • Wu S, Chen J, Zhou W, Iqbal J, Yao L (2018) A modified Logit model for assessment and validation of debris-flow susceptibility. Bull Eng Geol Env 78:4421–4438

    Article  Google Scholar 

  • Xiao T, Yin K, Yao T, Liu S (2019) Spatial prediction of landslide susceptibility using GIS-based statistical and machine learning models in Wanzhou County, Three Gorges Reservoir, China. Acta Geochimica 38:654–669

    Article  Google Scholar 

  • Yao T, Xue Y, Chen D, Chen F, Thompson L, Cui P, Koike T, Lau WKM, Lettenmaier D, Mosbrugger V, Zhang R, Xu B, Dozier J, Gillespie T, Gu Y, Kang S, Piao S, Sugimoto S, Ueno K, Wang L, Wang W, Zhang F, Sheng Y, Guo W, Ailikun YX, Ma Y, Shen SSP, Su Z, Chen F, Liang S, Liu Y, Singh VP, Yang K, Yang D, Zhao X, Qian Y, Zhang Y, Li Q (2019) Recent third pole’s rapid warming accompanies cryospheric melt and water cycle intensification and interactions between monsoon and environment: multidisciplinary approach with observations, modeling, and analysis. Bull Am Meteorol Soc 100:423–444

    Article  Google Scholar 

  • Yi Y, Zhang Z, Zhang W, Jia H, Zhang J (2020) Landslide susceptibility mapping using multiscale sampling strategy and convolutional neural network: A case study in Jiuzhaigou region. CATENA 195:104851

    Article  Google Scholar 

  • Zhane Y, Jie Y, Shiyuan X, Jiahong W (2011) Community-based scenario modelling and disaster risk assessment of urban rainstorm waterlogging. J Geograph Sci 21:274–284

    Article  Google Scholar 

  • Zhang L, Li Y, Zhang F, Chen L, Pan T, Wang B, Ren C (2020) Changes of winter extreme precipitation in Heilongjiang province and the diagnostic analysis of its circulation features. Atmos Res 245:105094

    Article  Google Scholar 

  • Zhao X, Chen W (2020) Optimization of computational intelligence models for landslide susceptibility evaluation. Remote Sens 12(14):2180

    Article  Google Scholar 

  • Zhao Q, Zou CH, Wang KF, Gao Q, Yao T (2019) Spatial and temporal distribution characteristics of drought and its influencing factors in Heilongjiang Province, China from 1956 To 2015. Appl Ecol Environ Res 17:2631–2650

    Article  Google Scholar 

  • Zhong C, Liu Y, Gao P, Chen W, Li H, Hou Y, Nuremanguli T, Ma H (2019) Landslide mapping with remote sensing: challenges and opportunities. Int J Remote Sens 41:1555–1581

    Article  Google Scholar 

  • Zhu AX, Miao Y, Liu J, Bai S, Zeng C, Ma T, Hong H (2019) A similarity-based approach to sampling absence data for landslide susceptibility mapping using data-driven methods. CATENA 183:104188

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported and funded by the National Key Research and Development Program of China (2016YFE0202400 and 2018YFC0809605), aiming to study the interaction between natural disasters and engineering infrastructure in cold regions. The authors are grateful for this support and would also like to thank the professors for providing the dates used in this study.

Author information

Authors and Affiliations

  1. School of Civil Engineering, Harbin Institute of Technology, No. 73 Huanghe Rd, Nangang District, Harbin, 150001, China

    Qiang Liu, Delong Huang & Aiping Tang

  2. School of Engineering and Technology, China University of Geosciences (Beijing), Beijing, 100083, China

    Xiaosheng Han

Authors
  1. Qiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Delong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aiping Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaosheng Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aiping Tang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest. Some or all data, models, or codes generated or used during the study are available from the corresponding author by request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Huang, D., Tang, A. et al. Model performance analysis for landslide susceptibility in cold regions using accuracy rate and fluctuation characteristics. Nat Hazards 108, 1047–1067 (2021). https://doi.org/10.1007/s11069-021-04719-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11069-021-04719-4

Keywords

Search

Navigation