Abstract
Considering the increasing number of landslides due to permafrost degradation, this paper reports a performance evaluation of three classical landslide susceptibility models applied to cold regions. A landslide inventory was first constructed through a historical survey and image interpretation. Ten causative factors of landslides were then chosen based on the available data and the local environment. Multicollinearity diagnosis and factor effectiveness test were employed to perform a factor analysis. Subsequently, three evaluation models based on the frequency ratio (FR), logistic regression (LR), and artificial neural network (ANN) were established. These models were applied to obtain landslide susceptibility maps, which were then analyzed and compared. The model performance was evaluated in terms of the accuracy rate and fluctuation characteristics. The results showed no multicollinearity issue between the factors employed. The annual temperature difference and frozen depth are two indispensable factors when assessing landslide susceptibility in cold regions. A comparison between the susceptibility maps generated using the three models showed that the FR model-generated map is most in line with the principle of disaster zoning and has the highest degree of conformity with actual landslide points, followed by the maps generated using the LR and ANN models. An accuracy analysis showed that the ANN model yields the highest AUC value in the training and test states, 0.957 and 0.863, respectively; however, these values were not optimal given the fluctuation. Moreover, the fluctuation in the non-landslide data was greater than that in the landslide data. The fluctuation results revealed the drawback of the AUC value in the analysis of the model performance. In other words, the non-landslide error often covers up the landslide error. This study provides a scientific guidance for evaluating the model performance and for assessing landslide disasters in cold regions.
Similar content being viewed by others
References
Al-Abadi AM, Al-Najar NA (2019) Comparative assessment of bivariate, multivariate and machine learning models for mapping flood proneness. Nat Hazards 100:461–491
Allen S, Huggel C (2013) Extremely warm temperatures as a potential cause of recent high mountain rockfall. Global Planet Change 107:59–69
Allen SK, Cox SC, Owens IF (2010) Rock avalanches and other landslides in the central Southern Alps of New Zealand: a regional study considering possible climate change impacts. Landslides 8:33–48
Barella CF, Sobreira FG, Zêzere JL (2018) A comparative analysis of statistical landslide susceptibility mapping in the southeast region of Minas Gerais state, Brazil. Bull Eng Geol Env 78:3205–3221
Blikra LH, Christiansen HH (2014) A field-based model of permafrost-controlled rockslide deformation in northern Norway. Geomorphology 208:34–49
Cantarino I, Carrion MA, Goerlich F, Martinez Ibañez V (2018) A ROC analysis-based classification method for landslide susceptibility maps. Landslides 16:265–282
Chae B-G, Park H-J, Catani F, Simoni A, Berti M (2017) Landslide prediction, monitoring and early warning: a concise review of state-of-the-art. Geosci J 21:1033–1070
Chen W, Shirzadi A, Shahabi H, Ahmad BB, Zhang S, Hong H, Zhang N (2017) A novel hybrid artificial intelligence approach based on the rotation forest ensemble and naïve Bayes tree classifiers for a landslide susceptibility assessment in Langao County, China. Geomat Nat Haz Risk 8:1955–1977
Chen W, Yan X, Zhao Z, Hong H, Bui DT, Pradhan B (2018) Spatial prediction of landslide susceptibility using data mining-based kernel logistic regression, naive Bayes and RBFNetwork models for the Long County area (China). Bull Eng Geol Environ 78(1):247–266
Delan X, Changlin G (2011) The characteristics of thegeological environment and geological disasters, Heilongjiang Province. Chin J Geol Hazard Control 22:101–106
Ding Y, Zhang S, Zhao L, Li Z, Kang S (2019) Global warming weakening the inherent stability of glaciers and permafrost. Sci Bull 64:245–253
Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, Marquéz JRG, Gruber B, Lafourcade B, Leitão PJ, Münkemüller T, McClean C, Osborne PE, Reineking B, Schröder B, Skidmore AK, Zurell D, Lautenbach S (2013) Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 36:27–46
Dou J, Yunus AP, Merghadi A, Shirzadi A, Nguyen H, Hussain Y, Avtar R, Chen Y, Pham BT, Yamagishi H (2020) Different sampling strategies for predicting landslide susceptibilities are deemed less consequential with deep learning. Sci Total Environ 720:137320
Fattahi H, Zandy Ilghani N (2020) Slope Stability Analysis Using Bayesian Markov Chain Monte Carlo Method. Geotech Geol Eng 38:2609–2618
Fell R, Corominas J, Bonnard C, Cascini L, Leroi E, Savage WZ (2008) Guidelines for landslide susceptibility, hazard and risk zoning for land-use planning. Eng Geol 102:99–111
Feng J, Li N, Zhang Z, Chen X (2017) How to apply the dependence structure analysis to extreme temperature and precipitation for disaster risk assessment. Theoret Appl Climatol 133:297–305
Gao J, Liu Y (2011) Climate warming and land use change in Heilongjiang Province, Northeast China. Appl Geogr 31:476–482
Guzzetti F, Reichenbach P, Cardinali M, Galli M, Ardizzone F (2005) Probabilistic landslide hazard assessment at the basin scale. Geomorphology 72:272–299
Haeberli W (2013) Mountain permafrost—research frontiers and a special long-term challenge. Cold Reg Sci Technol 96:71–76
Hong H, Miao Y, Liu J, Zhu AX (2019) Exploring the effects of the design and quantity of absence data on the performance of random forest-based landslide susceptibility mapping. CATENA 176:45–64
Huang Y, Zhao L (2018) Review on landslide susceptibility mapping using support vector machines. CATENA 165:520–529
Huggel C, Salzmann N, Allen S, Caplan-Auerbach J, Fischer L, Haeberli W, Larsen C, Schneider D, Wessels R (2010) Recent and future warm extreme events and high-mountain slope stability. Philos Trans A Math Phys Eng Sci 368:2435–2459
Kia MB, Pirasteh S, Pradhan B, Mahmud AR, Sulaiman WNA, Moradi A (2011) An artificial neural network model for flood simulation using GIS: Johor River Basin, Malaysia. Environ Earth Sci 67:251–264
Lan H, Derek Martin C, Lim CH (2007) RockFall analyst: A GIS extension for three-dimensional and spatially distributed rockfall hazard modeling. Comput Geosci 33:262–279
Lan H, Martin CD, Zhou C, Lim CH (2010) Rockfall hazard analysis using LiDAR and spatial modeling. Geomorphology 118:213–223
Lee YH, Kim JR, Kim D, Kang HB (2007) New Approach for Estimating Rock Slope Stability Using Logistic Regression Analysis. Transp Res Rec J Transp Res Board 2016:99–109
Li L, Lan H (2015) Probabilistic modeling of rockfall trajectories: a review. Bull Eng Geol Env 74:1163–1176
Li C, Ma T, Sun L, Li W, Zheng A (2011) Application and verification of a fractal approach to landslide susceptibility mapping. Nat Hazards 61:169–185
Li L, Lan H, Guo C, Zhang Y, Li Q, Wu Y (2016) A modified frequency ratio method for landslide susceptibility assessment. Landslides 14:727–741
Luo X, Lin F, Chen Y, Zhu S, Xu Z, Huo Z, Yu M, Peng J (2019) Coupling logistic model tree and random subspace to predict the landslide susceptibility areas with considering the uncertainty of environmental features. Sci Rep 9:15369
Palenzuela Baena JA, Scifoni S, Marsella M, De Astis G, Irigaray Fernández C (2019) Landslide susceptibility mapping on the islands of Vulcano and Lipari (Aeolian Archipelago, Italy), using a multi-classification approach on conditioning factors and a modified GIS matrix method for areas lacking in a landslide inventory. Landslides 16:969–982
Patton AI, Rathburn SL, Capps DM (2019) Landslide response to climate change in permafrost regions. Geomorphology 340:116–128
Pham VD, Nguyen Q-H, Nguyen H-D, Pham V-M, Vu VM, Bui Q-T (2020) Convolutional Neural Network—Optimized Moth Flame Algorithm for Shallow Landslide Susceptible Analysis. IEEE Access 8:32727–32736
Pourghasemi HR, Kornejady A, Kerle N, Shabani F (2020) Investigating the effects of different landslide positioning techniques, landslide partitioning approaches, and presence-absence balances on landslide susceptibility mapping. CATENA 187:104364
Pradhan AMS, Kim Y-T (2016) Spatial data analysis and application of evidential belief functions to shallow landslide susceptibility mapping at Mt. Umyeon, Seoul, Korea. Bull Eng Geol Env 76:1263–1279
Pradhan B, Lee S (2009) Regional landslide susceptibility analysis using back-propagation neural network model at Cameron Highland, Malaysia. Landslides 7:13–30
Pradhan B, Lee S (2010) Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modelling. Environ Model Softw 25:747–759
Reichenbach P, Rossi M, Malamud BD, Mihir M, Guzzetti F (2018) A review of statistically-based landslide susceptibility models. Earth Sci Rev 180:60–91
Sameen MI, Pradhan B, Bui DT, Alamri AM (2020) Systematic sample subdividing strategy for training landslide susceptibility models. CATENA 187:104358
Schaub Y, Haeberli W, Huggel C, Künzler M, Bründl M (2013) Landslides and New Lakes in Deglaciating Areas: A Risk Management Framework. Springer, Berlin
Shafapour Tehrany M, Shabani F, Neamah Jebur M, Hong H, Chen W, Xie X (2017) GIS-based spatial prediction of flood prone areas using standalone frequency ratio, logistic regression, weight of evidence and their ensemble techniques. Geomat Nat Haz Risk 8:1538–1561
Shao X, Ma S, Xu C, Zhou Q (2020) Effects of sampling intensity and non-slide/slide sample ratio on the occurrence probability of coseismic landslides. Geomorphology 363:107222
Sharma S, Mahajan AK (2019) Information value based landslide susceptibility zonation of Dharamshala region, northwestern Himalaya, India. Spat Inf Res 27:553–564
Shirzadi A, Saro L, Hyun Joo O, Chapi K (2012) A GIS-based logistic regression model in rock-fall susceptibility mapping along a mountainous road: Salavat Abad case study, Kurdistan. Iran Nat Hazards 64:1639–1656
Soma AS, Kubota T, Mizuno H (2019) Optimization of causative factors using logistic regression and artificial neural network models for landslide susceptibility assessment in Ujung Loe Watershed, South Sulawesi Indonesia. J Mt Sci 16:383–401
Steger S, Brenning A, Bell R, Petschko H, Glade T (2016) Exploring discrepancies between quantitative validation results and the geomorphic plausibility of statistical landslide susceptibility maps. Geomorphology 262:8–23
Tehrany MS, Shabani F, Jebur MN, Hong H, Chen W, Xie X (2017) GIS-based spatial prediction of flood prone areas using standalone frequency ratio, logistic regression, weight of evidence and their ensemble techniques. Geom Nat Hazards Risk 8(2):1538–1561
Thai Pham B, Shirzadi A, Tien Bui D, Prakash I, Dholakia MB (2018) A hybrid machine learning ensemble approach based on a Radial Basis Function neural network and Rotation Forest for landslide susceptibility modeling: a case study in the Himalayan area, India. Int J Sediment Res 33(2):157–170
Thi Ngo PT, Panahi M, Khosravi K, Ghorbanzadeh O, Kariminejad N, Cerda A, Lee S (2021) Evaluation of deep learning algorithms for national scale landslide susceptibility mapping of Iran. Geosci Front 12:505–519
Tien Bui D, Tuan TA, Klempe H, Pradhan B, Revhaug I (2016) Spatial prediction models for shallow landslide hazards: a comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides 13:361–378
Wenmei Z (2015) Study on physical and mechanical properties of permafrost at high altitudes. Forest Eng 31:128–131
Wu S, Chen J, Zhou W, Iqbal J, Yao L (2018) A modified Logit model for assessment and validation of debris-flow susceptibility. Bull Eng Geol Env 78:4421–4438
Xiao T, Yin K, Yao T, Liu S (2019) Spatial prediction of landslide susceptibility using GIS-based statistical and machine learning models in Wanzhou County, Three Gorges Reservoir, China. Acta Geochimica 38:654–669
Yao T, Xue Y, Chen D, Chen F, Thompson L, Cui P, Koike T, Lau WKM, Lettenmaier D, Mosbrugger V, Zhang R, Xu B, Dozier J, Gillespie T, Gu Y, Kang S, Piao S, Sugimoto S, Ueno K, Wang L, Wang W, Zhang F, Sheng Y, Guo W, Ailikun YX, Ma Y, Shen SSP, Su Z, Chen F, Liang S, Liu Y, Singh VP, Yang K, Yang D, Zhao X, Qian Y, Zhang Y, Li Q (2019) Recent third pole’s rapid warming accompanies cryospheric melt and water cycle intensification and interactions between monsoon and environment: multidisciplinary approach with observations, modeling, and analysis. Bull Am Meteorol Soc 100:423–444
Yi Y, Zhang Z, Zhang W, Jia H, Zhang J (2020) Landslide susceptibility mapping using multiscale sampling strategy and convolutional neural network: A case study in Jiuzhaigou region. CATENA 195:104851
Zhane Y, Jie Y, Shiyuan X, Jiahong W (2011) Community-based scenario modelling and disaster risk assessment of urban rainstorm waterlogging. J Geograph Sci 21:274–284
Zhang L, Li Y, Zhang F, Chen L, Pan T, Wang B, Ren C (2020) Changes of winter extreme precipitation in Heilongjiang province and the diagnostic analysis of its circulation features. Atmos Res 245:105094
Zhao X, Chen W (2020) Optimization of computational intelligence models for landslide susceptibility evaluation. Remote Sens 12(14):2180
Zhao Q, Zou CH, Wang KF, Gao Q, Yao T (2019) Spatial and temporal distribution characteristics of drought and its influencing factors in Heilongjiang Province, China from 1956 To 2015. Appl Ecol Environ Res 17:2631–2650
Zhong C, Liu Y, Gao P, Chen W, Li H, Hou Y, Nuremanguli T, Ma H (2019) Landslide mapping with remote sensing: challenges and opportunities. Int J Remote Sens 41:1555–1581
Zhu AX, Miao Y, Liu J, Bai S, Zeng C, Ma T, Hong H (2019) A similarity-based approach to sampling absence data for landslide susceptibility mapping using data-driven methods. CATENA 183:104188
Acknowledgements
This research was supported and funded by the National Key Research and Development Program of China (2016YFE0202400 and 2018YFC0809605), aiming to study the interaction between natural disasters and engineering infrastructure in cold regions. The authors are grateful for this support and would also like to thank the professors for providing the dates used in this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest. Some or all data, models, or codes generated or used during the study are available from the corresponding author by request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, Q., Huang, D., Tang, A. et al. Model performance analysis for landslide susceptibility in cold regions using accuracy rate and fluctuation characteristics. Nat Hazards 108, 1047–1067 (2021). https://doi.org/10.1007/s11069-021-04719-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11069-021-04719-4