# Spatial prediction models for shallow landslide hazards: a comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree

- 5.1k Downloads
- 201 Citations

## Abstract

Preparation of landslide susceptibility maps is considered as the first important step in landslide risk assessments, but these maps are accepted as an end product that can be used for land use planning. The main objective of this study is to explore some new state-of-the-art sophisticated machine learning techniques and introduce a framework for training and validation of shallow landslide susceptibility models by using the latest statistical methods. The Son La hydropower basin (Vietnam) was selected as a case study. First, a landslide inventory map was constructed using the historical landslide locations from two national projects in Vietnam. A total of 12 landslide conditioning factors were then constructed from various data sources. Landslide locations were randomly split into a ratio of 70:30 for training and validating the models. To choose the best subset of conditioning factors, predictive ability of the factors were assessed using the *Information Gain Ratio* with 10-fold cross-validation technique. Factors with null predictive ability were removed to optimize the models. Subsequently, five landslide models were built using support vector machines (SVM), multi-layer perceptron neural networks (MLP Neural Nets), radial basis function neural networks (RBF Neural Nets), kernel logistic regression (KLR), and logistic model trees (LMT). The resulting models were validated and compared using the receive operating characteristic (ROC), Kappa index, and several statistical evaluation measures. Additionally, Friedman and Wilcoxon signed-rank tests were applied to confirm significant statistical differences among the five machine learning models employed in this study. Overall, the MLP Neural Nets model has the highest prediction capability (90.2 %), followed by the SVM model (88.7 %) and the KLR model (87.9 %), the RBF Neural Nets model (87.1 %), and the LMT model (86.1 %). Results revealed that both the KLR and the LMT models showed promising methods for shallow landslide susceptibility mapping. The result from this study demonstrates the benefit of selecting the optimal machine learning techniques with proper conditioning selection method in shallow landslide susceptibility mapping.

## Keywords

Landslide GIS Support vector machines Neural network Kernel logistic regression Decision trees Son La hydropower## Introduction

Understanding landslide mechanisms and mapping areas susceptible to landslides is essential for land use planning and may be considered as a standard tool that supports decision-making activities. However, due to the complex nature of landslides such as soil condition, root strength, bedrock, topography, hydrology, and human activities (Wu and Sidle 1995), producing a reliable spatial prediction of landslides is still a challenging task. The best landslide model for an area depends not only on the quality of the data used (Jebur et al. 2014; Tien Bui 2012) but also strongly on the employed modeling approaches (Yilmaz 2009). To address this, a broad range of methods and techniques have been proposed from different points of view to understand their controlling factors and to predict their spatial-temporal distribution. These methods and techniques vary from simple expert knowledge to sophisticated mathematical procedures (Chung and Fabbri 2008; Cross 2002; Erener and Düzgün 2010; Fernández et al. 2003; Gokceoglu and Sezer 2009; Pradhan et al. 2010a, b) and they can be grouped into physically based and statistics-based correlation analyses.

Physically based methods (Duc 2013; Montgomery and Dietrich 1994; Thanh and De Smedt 2014; Van Westen and Terlien 1996) require detailed data of geotechnical engineering and engineering geological aspect of the slope failure at site-specific locations in localized scale. These methods generally provide accurate results because of its site-specific and data dependency nature. Factor of safety is generally used as the index of stability by taking into account stabilizing and destabilizing factors for a number of scenarios (Gokceoglu and Aksoy 1996; Jia et al. 2012). The physical-based models do not require landslide inventories and are most suitable for small and well-monitored slope failure sites. These methods are quite expensive and not practical for large-scale areas where detailed information on many geomorphological and geological factors is needed (Van Westen and Terlien 1996).

Statistical-based correlation analysis assumes conditions that lead to slope failure in the past and present are likely to cause landslides in the future, therefore landslide inventory map and various conditioning factors should be collected in the first stage. Then, statistical models are trained and cross-validated using the two datasets and the resulting models are applied to estimate landslide occurrence probabilities. To obtain reliable results, statistical models require collection of large amounts of data which is time consuming and involves various complex processes (Tien Bui et al. 2012c). In the literature, the most common statistical and machine learning models used in landslide modeling are logistic regression (Atkinson and Massari 1998; Costanzo et al. 2014; Felicisimo et al. 2013; Kavzoglu et al. 2014a; Lee 2005; Pradhan and Lee 2010a; Tien Bui et al. 2011; Tunusluoglu et al. 2008), discriminant analysis (Guzzetti et al. 2006b), fuzzy logic (Akgun et al. 2012; Ercanoglu and Gokceoglu 2002; Lee 2007; Pourghasemi et al. 2012a; Pradhan 2011; Tien Bui et al. 2012e), artificial neural networks (Gomez and Kavzoglu 2005; Lee et al. 2003; Pradhan et al. 2010a, b; Tien Bui et al. 2012c; Yilmaz 2009), support vector machines (Brenning 2005; Kavzoglu et al. 2014b; Tien Bui et al. 2012a, b; Yao et al. 2008; Yilmaz 2010), decision tree (Nefeslioglu et al. 2010; Pradhan 2012; Tien Bui et al. 2012a, 2013a), and neuro-fuzzy (Pradhan et al. 2010b; Sezer et al. 2011; Tien Bui et al. 2012d). Early review on advantages and disadvantages of different approaches can be found in some published articles, for example, Chacon et al. (2006).

Despite many efforts that have been made in landslide susceptibility and hazard modeling, there still lies dispute on which method or technique is the best for the prediction of landslide prone areas (Carrara and Pike 2008). Even 1 or 2 % of the increment of the prediction accuracy could control the resulting landslide susceptibility zones (Jebur et al. 2014; Pourghasemi et al. 2012b; Pradhan 2012; Tien Bui et al. 2012a, b, 2013a, 2014), and therefore it is necessary to accurately predict these zones with high-performance-based models.

The recent development in statistical algorithms, machine learning, and geographic information systems (GIS) have continuously introduced new and powerful techniques for landslide modeling (Vorpahl et al. 2012). Some of these methods were reported to outperform the conventional methods (Tien Bui et al. 2013b). Literature review shows that many advanced machine learning techniques such as kernel logistic regression and logistic model trees have seldom been explored for landslide modeling. Therefore, the investigation of these methods and techniques including their comparison with conventional methods is highly necessary to acquire adequate background to reach some reasonable conclusions. More importantly, there is still a lack of a quantitative-systematic comparison of available machine learning techniques in landslide susceptibility modeling.

The main objective of this study is therefore to evaluate and compare several state-of-the-art machine learning techniques for shallow landslide susceptibility modeling. Feature selection is adopted using the *Information Gain Ratio* with 10-fold cross-validation technique which is seldom used in landslide modeling to assess the predictive capability of the conditioning factors. Then factors with null predictive ability were excluded to optimize landslide models. The receive operating characteristics (ROC), Kappa index, and various statistical evaluation measures were used for the assessment, validation, and comparison of the resulting models in order to choose the best model in this study. Finally, Friedman and Wilcoxon signed-rank tests were used to confirm significant statistical differences among the five models.

## Study area and spatial database

### Description of study area

^{2}and covers the whole area of Quynh Nhai district and most of the area of Muong La and Thuan Chau districts. It is located in the northern part of the Son La province that belongs to the Northwest mountainous area of Vietnam, between longitudes 103°28′E and 104°12′E, and latitudes 21°21′N and 22°02′N. It is an area with a high-density network of rivers and waterfalls being an abundant water source with a heavy flow rate. These conditions are suitable for the hydropower development in the region. The study area has the largest hydropower plant in Vietnam with an installed capacity of 2400 MW.

The topography is hilly with low relief mountains. The highest altitude is 2879 m above sea level, situated on the strip of the Hoang Lien Son Mountain range in the north. The lowest point is 70 m above sea level. The topographic inclination follows the NW–SE direction. Areas with slope greater than 15° cover about 76.47 % of the total area while areas with slopes between 15 and 35° account for 56 %, and areas with slope over 35° cover 20.44 %. On the other hand, areas with slope less than 8° only cover about 12.52 %.

The study area is situated in the tropical monsoon region with two separate seasons: (1) hot, rainy in the summer and (2) cold, dry in the winter. The rainy season is normally from April to September. The rainfall is concentrated in July, August, and September accounting for 80–92 % of the total annual rainfall with 300–700 mm/month. The largest daily rainfall is greater than 100 mm during the rainy season. The dry season is generally from October to March with average rainfall less than 50 mm/month. December and January are the two months that have irregular rain with rainfall between 10 and 30 mm/month. In the dry season, the largest daily rainfall ranges between 30 and 80 mm/day. The annual average rainfall ranges between 1200 and 1600 mm with average of 123 rainy days. The annual average temperature is 21.4 °C, i.e., the highest and lowest average temperatures are 27 and 16 °C, respectively. The annual average moisture is 81 %.

Natural vegetation type is the jungle class constituted by timber trees with closed canopy. The study area comprises forest land (16.7 %), woodlands sparse and shrubs land (38.3 %), perennial crop land (1.1 %), agricultural land (42.8 %), and water surface (1.1 %). The descriptive statistical results show that 53.1 % of landslides have occurred in agricultural land, followed by woodlands sparse and shrubs land (23.5 %), forest land (20.4 %), and perennial crop land (3 %).

The study area is mainly resided by an ethnic minority with traditional farming style with unplanned use of land. These traditional human activities have caused negative impacts to the natural environment.

### Spatial database

The first important step in the landslide susceptibility mapping is to identify landslide locations that occurred in the past and present (Hungr et al. 2005; Jiménez-Perálvarez et al. 2011). In this study, a detailed and reliable landslide inventory map (Fig. 1) was constructed from two main sources: (1) the landslide database from the national project of Vietnam constructed by Yem (2006) with 26 landslide locations and (2) the landslide database inventory map constructed by Dan et al. (2011) with 72 landslide locations. A total of 98 shallow rotational landslides that have occurred during the last 30 years were registered. These landslides are mostly distributed on the banks of the Da river, the Son La hydropower lake, and near the road system (Tuan and Dan 2012). The occurrence dates of these landslide locations are mostly unknown.

^{3}(Dan et al. 2011). Many landslides occurred also in the tropical storm Rammasun from 19th to 22nd of July 2014, where the accumulated measured rainfall in these 4 days at the Quynh Nhai station was 255.3 mm. A total of 62 families were relocated due to these landslides. Figure 3 shows two pictures of new landslide locations in the study area.

Around 55 % of the landslides occurred in Muong Trai, Yen Chau, and Suoi Bang suites where the main lithologies are conglomerate, sandstone, aleurolite, and gritstone. Around 23 % of the landslides are distributed in Vien An, Nam Mu, and Dong Giao formations where clay shale, siltstones, light grey, and basic effusives are the dominant lithologies. Most landslides occurred in the saprolite on carbonate rocks (44 %) and the ferrosiallite on carbonate rocks (37 %). Only around 7 % of the landslides occurred in the Quaternary deposit.

In the previous study, Tuan and Dan (2012) have investigated and analyzed the relationship between landslide occurrences and conditioning factors for the study area, and based on their findings, 12 landslide conditioning factors [e.g., slope, aspect, altitude, relief amplitude, topographic wetness index (TWI), stream power index (SPI), sediment transport index (STI), lithology, fault density, land use, and rainfall] were selected in this study.

The lithology and fault density maps were constructed based on the Geological and Mineral Resources maps in 1:200,000 scale (General Department of Geology and Minerals of Vietnam 2000). This is the only geological map available for the study area. The lithology map was constructed with seven groups based on clay composition, degree of weathering, and estimated strength and density (Van et al. 2006) (Fig. 4h). The fault density map with six classes (Fig. 4i) was constructed. The weathering map (Fig. 4j) was constructed based on slope, lithology, and field survey by Dan et al. (2011).

The land use map with five classes (Fig. 4k) was constructed based on the forest map at the scale 1:50,000 (Forest Inventory and Planning Institute 2005) and then was updated using field survey data (Dan et al. 2011). The rainfall map (Fig. 4l) was constructed from the annual average of rainfall data for the period from 1990 to 2010 using the Inverse Distance Weighed method (Tuan and Dan 2012). The precipitation data were sourced from the Institute of Meteorology and Hydrology in Vietnam.

## Methodology

### Preparation of training and validation datasets

It is important to emphasize that landslide susceptibility mapping is considered to be a binary classification in which landslide index is separated into two classes, i.e., landslide and non-landslide. Landslide pixels are assigned a value “1” while non-landslide pixels are assigned a value “0”. The resulting landslide susceptibility indexes are expressed in terms of probability of occurrence and vary between 0 and 1 (0–100 %).

Based on the initial set of the conditioning factors, the process of converting continuous variables into categorical classes were carried out using expert opinions to define the class intervals (Tien Bui et al. 2011). Frequency ratio value was calculated for all categorical classes and then rescaled in the range 0.01 to 0.99. For landslide susceptibility modeling, the landslide locations should be divided into training and validation dataset. One set will be used for building models and the other will be used to validate the models and to confirm their accuracy (Chung and Fabbri 2008). Since the dates of the landslide locations are mostly unknown, we randomly split the landslide locations in this study in two subsets with a 70:30 ratio. The first one (68 landslide grid cells) is used for building models whereas the second one (30 landslide grid cells) is used for the model validation, respectively. The same number of non-landslide grid cells was randomly sampled from the landslide-free area (Tien Bui et al. 2012a, c, d). In the next step, values for the 12 landslide conditioning factors were extracted to build the training and validation data.

### Landslide conditioning factor analysis

#### Correlation analysis

In landslide studies, multicollinearity refers to the non-independence of conditioning factors that may occur in datasets due to their high correlation, thus resulting in erroneous system analysis (Dormann et al. 2013). To quantify multicollinearity, several methods have been proposed such as Pearson’s correlation coefficients (Booth et al. 1994), the variance decomposition proportions (Schuerman 1983), the conditional index (Belsley 1991), and the variance inflation factors (VIF) and tolerances (Hair et al. 2009; Liao and Valliant 2012). The VIF and tolerances methods are commonly used to check multicollinearity of conditioning factors in landslide studies (Tien Bui et al. 2011) whereas Pearson’ correlation coefficients method is widely used in other fields (Dormann et al. 2013).

*Sl*) and aspect (

*As*), is assessed using the Pearson method and defined as the covariance of the two factors divided by the product of their standard deviations (Eq. 1). Pearson’s correlation values larger than 0.7 indicate high collinearity (Booth et al. 1994).

*Sl*.

#### Factor selection based on the information gain ratio

In landslide modeling, all conditioning factors in the initial set may not have the equal predictive ability, and even in some cases some of them may cause a noise that reduces prediction capability of the resulting models. Therefore, predictive abilities of the conditioning factor should be quantified and factors with low or null predictiveness should be removed. This will result in a more accurate prediction of the resulting models (Martínez-Álvarez et al. 2013).

There are several techniques to quantifying the predictive ability of factors such as *Fuzzy-Rough sets* (Dubois and Prade 1990), *Relief* (Kononenko 1994), *Information Gain* (Hunter et al. 1966), and *Information Gain Ratio* (Quinlan 1993). *Information Gain* is based on information theory that tracks the decrease in entropy to quantify the importance of factors and is considered as the standard technique for measuring the predictive abilities of factors in data mining (Witten et al. 2011). However, *Information Gain* has a natural bias that tends to favor attributes with many possible values and, thus, may lead to a low prediction capability of resulting models (Xiaomeng and Borgelt 2004). To overcome this issue, Quinlan (1993) proposed *Information Gain Ratio* where higher *Information Gain Ratio* indicates a higher predictive ability for the models. *Information Gain Ratio* was used in this study.

*S*consists of

*n*input samples,

*n*(

*L*

_{ i },

*S*) is the number of samples in the training data

*S*belonging to the class

*L*

_{ i }(landslide, non-landslide). The information (entropy) that needs to classify

*S*is calculated as:

*S*into (

*S*

_{1},

*S*

_{2},…,

*S*

_{ m }) regarding the landslide conditioning factor

*A*is estimated as:

*Information Gain Ratio*for a certain landslide conditioning factor

*A*is computed as:

*SplitInfo*represents the potential information generated by dividing the training data

*S*into

*m*subsets.

*SplitInfo*is calculated as:

### Spatial prediction modeling of landslides

In this study, a total of five classifiers were selected and compared including support vector machines (SVM), multilayer perceptron artificial neural networks (MLP Neural Nets), radial basis function artificial neural networks (RBF Neural Nets), kernel logistic regression (KLR), and logistic model trees (LMT). The analysis was carried out using the R programming environment, Weka 3.7, Matlab 7.11, and ArcGIS 10.

#### Support vector machines

Support vector machines (SVM) is a machine learning method based on statistical learning theory that transforms original input space into a higher-dimensional feature space to find an optimal separating hyperplane (Abe 2010; Kavzoglu and Colkesen 2009; Vapnik 1998). SVM has been used for landslide modeling with reports that the prediction capability of models derived outperforms those obtained by conventional methods (Ballabio and Sterlacchini 2012; Pradhan 2012; Tien Bui et al. 2012a). The performance of a SVM model is affected by the use of the kernel functions such as linear, polynomial, sigmoid, and radial basis function (RBF). However, RBF kernel is one of the most commonly used landslide modeling and was selected in this study.

The behavior of SVM using RBF kernel is influenced by the kernel width (*γ*) and the regularization (*C*) parameters; therefore, determination of the best pairs of parameters for the study was carried out. The optimal pairs of parameters were obtained using the grid search method which is widely used and still considered as one of the most reliable optimization techniques (Kavzoglu and Colkesen 2009; Zhuang and Dai 2006). The best *C* and *γ* are 0.8 and 0.95, respectively, and obtained using the training data. With the best pair of parameters, the SVM model was built using the training data and then applied to calculate landslide susceptibility indexes for the entire study area.

#### Artificial neural networks

An artificial neural network is defined as a set of interconnected nodes which can be suitably used for the modeling of problems where relationships between causal factors and responses are complex such as landslides. Various algorithms for neural networks have been proposed in the literature. Multi-layer perceptron (MLP Neural Nets) and radial basis function (RBF Neural Nets) are the most widely used ANNs in landslide modeling.

The performance of the MLP Neural Nets is influenced by their structure, activation functions, and the way connection weights are updated (Haykin 1998). The ANNs generally consist of an input layer, one or more hidden layers, and an output layer (Kavzoglu and Mather 2003). The number of input neurons is the same as the number of selected landslide conditioning factors whereas the number of hidden neurons is determined based on specific training data. Connection weights between the input neuron and the hidden neurons, the hidden neurons, and the outputs were first initialized and then updated using the back-propagation algorithm with two phases, i.e., forward and backward propagation. In the forward phase, the input is propagated forward through the layers resulting in a response at the output layer. The output response values are compared to target values, and the difference is estimated. In the backward phase, the connection weights were updated to minimize the difference.

The RBF Neural Nets are a popular alternative to the MLP Neural Nets and are extremely useful in solving complex problems such as landslides. However, they differ in the way that the hidden units perform computations (Witten et al. 2011). Although RBF Neural Nets also have three layers, they have only one hidden layer which is referred to as the RBF units. The main function of the RBF units is to cluster the input data to reduce its dimensionality and to transform the data to a new space (Gil and Johnsson 2010). The learning procedure of the RBF Neural Nets is carried out in two phases: (1) the numbers of clusters (hidden neurons) are calculated using the K-means algorithm and (2) optimal estimation of the kernel parameters.

#### Kernel logistic regression

*n*input samples (

*x*

_{ i },

*L*

_{ i }) with

**x**_{ i }∈

*R*

^{ n },

**L**_{ i }∈ {0, 1}.

*denotes a vector of the input space consisting of slope, aspect, altitude, relief amplitude, TWI, SPI, STI, lithology, fault density, weathering, land use, and rainfall. The two classes {0, 1} represent non-landslide and landslide. The aim of KLR is to find a discriminant function (Eq. 5) which could separate the two classes, landslide and non-landslide.*

**x***p*is the logistic function that ranges between 0 and 1,

*K*(

**x**_{ i },

**x**_{ j }) is the kernel function that satisfies Mercer’s condition (Mercer 1909),

*b*is the intercept, and

*α*

_{ i }is a vector of dual parameters.

*δ*is a tuning parameter.

Using the grid search method, the best tuning parameter and regularized parameter for KLR obtained with the training data are 0.015 and 0.03, respectively.

#### Logistic model tree

Decision tree is a hierarchical model to recursively split landslide conditioning factors into two classes, landslide and non-landslide, in terms of probability. The main advantage of the decision tree is the capability of decomposing complex problems into simpler issues with decision rules and applies the same strategy to a new problem. The ongoing development of machine learning has resulted in new powerful decision tree algorithms such as the logistic model tree (LMT) (Gama 2004) in which leaf nodes are replaced by a regression plane instead of a constant value (Witten et al. 2011).

The LMT is a combination of the C4.5 decision tree (Quinlan 1993) and logistic regression functions in which the *information gain* is used for splitting, and the LogitBoost algorithm (Landwehr et al. 2005) is used for fitting the logistic regression functions at a tree node. To prevent the problem of over-fitting of the built LMT, the CART algorithm (Breiman et al. 1984) is used for pruning.

**C**_{ i }(landslide or non-landslide) as follows (Doetsch et al. 2009):

*D*is the number of landslide conditioning factors and

*β*

_{ i }is the coefficient of the

*i*-th component in the input vector

*x*. The posterior probabilities in the leaf nodes of the LMT is computed using the linear logistic regression method (Landwehr et al. 2005)

*C*is the number of classes and the least-square fits

*L*

_{ C }(

*) are transformed such that ∑*

**x**_{ C = 1}

^{ C }

*L*

_{ C }(

*) = 0.*

**x**To determine the minimum number of instances per node of LMT, a test on the training dataset versus classification accuracy was carried out and ten instances were selected.

### Accuracy assessment and comparison

#### The receiver operating characteristics curve and Kappa index

The performance of the landslide susceptibility models can be assessed using the receiver operating characteristic (ROC) curve analysis. The ROC curve is a graph based on the *Sensitivity* and *Specificity* with various cut-off thresholds. For quantitative comparison, the area under the ROC curves (AUC) that is the statistic summary of the overall performance of the landslide models is used. AUC can be interpreted as the probability that the classifier will correctly rank a randomly chosen landslide pixel to be more indicative of a landslide than a randomly chosen non-landslide. An AUC value of 1 indicates a perfect model that correctly classified all landslide and non-landslide pixels, whereas when AUC is equal to 0, it indicates a non-informative model (Walter 2002). In order to test the significance of one classification scheme producing a higher AUC than another (Bradley 1997), the standard error of the AUC was used. The smaller the standard error, the better it is for the model.

The reliability of the landslide models could be measured using Kappa index (*κ*) (Cohen 1960; Saito et al. 2009; Tien Bui et al. 2012a; Van Den Eeckhaut et al. 2009). Kappa index explains the ability of the landslide models to classify the landslide pixels (Guzzetti et al. 2006a) and is calculated as the proportion of observed agreement beyond that expected by chance. According to Landis and Koch (1977), the strength of agreement given the Kappa magnitude is for 0.8–1.0 almost perfect, 0.6–0.8 substantial, 0.4–0.6 moderate, 0.2–0.4 fair, 0–0.2 slight, and ≤0 poor.

#### Quality parameters and accuracy measure

*Accuracy*,

*Sensitivity*,

*Specificity*,

*Positive predictive value*, and

*Negative predictive value. Accuracy*is the proportion of landslide and non-landslide pixels that the resulting models correctly classified;

*Sensitivity*is the proportion of landslide pixels that are correctly classified as landslide occurrences;

*Specificity*is the proportion of the non-landslide pixels that are correctly classified as non-landslide. The

*Positive predictive value*is the probability of pixels that are correctly classified as landslide and the

*Negative predictive value*is the probability of pixels that are correctly classified as non-landslide.

*true positive*) and TN (

*true negative*) are the number of pixels that are correctly classified whereas FP (

*false positive*) and FN (

*false negative*) are the numbers of pixels erroneously classified.

#### Inferential statistics

Parametric and non-parametric statistical procedures could be used to evaluate whether a new landslide model is considered better than the other. The first one is most suitable for data that are normally distributed with equal variances (D’Arco et al. 2012) whereas the second one may be used regardless of statistical assumptions (Derrac et al. 2011). A non-parametric test such as the Friedman test (Friedman 1937) requires no previous knowledge for the used data and still is valid even if the data are normally distributed (Martínez-Álvarez et al. 2013) and was selected in this study.

The Friedman test is considered as one of the most important non-parametric tests for multiple comparisons to detect significant differences between the behaviors of two or more models (Beasley and Zumbo 2003). The Friedman test has a null hypothesis, i.e., there are no differences between the performances of the landslide models. The *p* value, which is the probability of rejecting the null hypothesis if the hypothesis is true, is then estimated for each of the models. The higher the *p* value, the more likely that the null hypothesis is not true.

#### Relative importance of the landslide conditioning factors

Relative importance of the conditioning factors to a landslide model is affected by the use of methods and evaluation criteria. Therefore, factors that have a high contribution in a particular model may be useless for another and vice versa, thus the importance of a conditioning factor may represent a great variation. In this study, the relative importance and contribution of each conditioning factor was determined by excluding that factor and then calculating the overall accuracy of the model (Tien Bui et al. 2012b). The difference of the overall accuracy between these models indicates the quantitative importance of the factors.

## Results and analysis

### Landslide conditioning factor analysis

Multicollinearity analysis for the landslide conditioning factors

Number | Landslide conditioning factor | Tolerance | VIF |
---|---|---|---|

1 | Slope | 0.653 | 1.531 |

2 | Aspect | 0.900 | 1.112 |

3 | Altitude | 0.543 | 1.842 |

4 | Relief amplitude | 0.464 | 2.154 |

5 | TWI | 0.795 | 1.258 |

6 | SPI | 0.605 | 1.653 |

7 | STI | 0.496 | 2.014 |

8 | Lithology | 0.600 | 1.667 |

9 | Fault density | 0.907 | 1.103 |

10 | Weathering | 0.324 | 3.085 |

11 | Land use | 0.780 | 1.283 |

12 | Rainfall | 0.853 | 1.172 |

Pearson correlation between pairs of landslide conditioning factors

Slope | Aspect | Altitude | RA | TWI | SPI | STI | LT | FD | WT | LU | |
---|---|---|---|---|---|---|---|---|---|---|---|

Slope | 1 | ||||||||||

Aspect | 0.040 | 1 | |||||||||

Altitude | 0.382 | 0.118 | 1 | ||||||||

RA | 0.370 | −0.016 | 0.539 | 1 | |||||||

TWI | 0.044 | 0.108 | 0.031 | −0.089 | 1 | ||||||

SPI | 0.245 | 0.072 | 0.013 | 0.138 | 0.314 | 1 | |||||

STI | 0.448 | 0.110 | 0.175 | 0.199 | 0.335 | 0.594 | 1 | ||||

LT | 0.174 | 0.181 | 0.144 | 0.223 | −0.054 | −0.009 | −0.026 | 1 | |||

FD | 0.113 | 0.120 | 0.237 | 0.093 | 0.067 | 0.068 | 0.123 | −0.041 | 1 | ||

WT | 0.389 | 0.042 | 0.562 | 0.669 | −0.078 | 0.078 | 0.128 | 0.542 | 0.131 | 1 | |

LU | 0.176 | 0.062 | 0.149 | 0.286 | −0.107 | 0.194 | 0.229 | 0.126 | −0.016 | 0.254 | 1 |

RF | 0.098 | 0.042 | 0.164 | 0.247 | −0.013 | 0.066 | 0.127 | 0.048 | −0.020 | 0.207 | 0.332 |

*Information Gain Ratio*. Table 3 shows

*Information Gain Ratio*calculated for a total of 12 landslide conditioning factors in this study with the average merit as the average

*Information Gain Ratio*and its standard deviation using 10-fold cross-validation. It could be observed that the highest average merit is for altitude (0.546). It is followed by slope (0.413), weathering (0.389), relief amplitude (0.377), aspect (0.1), lithology (0.038), rainfall (0.029), and SPI (0.028). In contrast, four conditioning factors (fault density, land use, TWI, and STI) have “0” average merit, which indicates null predictiveness to the landslide occurrence. Therefore, the inclusion of the four factors may make a noise that negatively influences the resulting models, and thus they were removed from the analysis.

Average *Information Gain Ratio* for the landslide conditioning factors

Number | Conditioning factor | Average merit | Standard deviation |
---|---|---|---|

1 | Altitude | 0.546 | ±0.027 |

2 | Slope | 0.413 | ±0.023 |

3 | Weathering | 0.389 | ±0.028 |

4 | Relief amplitude | 0.377 | ±0.026 |

5 | Aspect | 0.100 | ±0.152 |

6 | Lithology | 0.038 | ±0.113 |

7 | Rainfall | 0.029 | ±0.088 |

8 | SPI | 0.028 | ±0.084 |

9 | Fault density | 0 | 0 |

10 | Land use | 0 | 0 |

11 | TWI | 0 | 0 |

12 | STI | 0 | 0 |

### Model results and analysis

Model performance

Parameter | SVM | MLP Neural Nets | RBF Neural Nets | KLR | LMT |
---|---|---|---|---|---|

True positive | 54 | 65 | 53 | 58 | 54 |

True negative | 58 | 52 | 56 | 60 | 56 |

False positive | 14 | 3 | 15 | 10 | 14 |

False negative | 10 | 16 | 12 | 8 | 12 |

Positive predictive value (%) | 79.41 | 95.59 | 77.94 | 85.29 | 79.41 |

Negative predictive value (%) | 85.29 | 76.47 | 82.35 | 88.24 | 82.35 |

Sensitivity (%) | 84.38 | 80.25 | 81.54 | 87.88 | 81.82 |

Specificity (%) | 80.56 | 94.55 | 78.87 | 85.71 | 80.00 |

Accuracy (%) | 82.35 | 86.03 | 80.15 | 86.76 | 80.88 |

Area under the ROC curves and Kappa index for the five landslide models on training data

Number | Landslide models | AUC | SE | 95 % CI | Kappa index |
---|---|---|---|---|---|

1 | SVM | 0.907 | 0.025 | 0.845–0.950 | 0.647 |

2 | MLP Neural Nets | 0.934 | 0.021 | 0.878–0.969 | 0.721 |

3 | RBF Neural Nets | 0.868 | 0.030 | 0.800–0.920 | 0.603 |

4 | KLR | 0.941 | 0.018 | 0.887–0.974 | 0.735 |

5 | LMT | 0.904 | 0.025 | 0.841–0.947 | 0.618 |

The highest positive predictive value is for MLP Neural Nets model (95.59 %) indicating the probability that the model correctly classifies pixels in the landslide class in 95.59 % of the cases. It is followed by the KLR model (85.29 %), the SVM model (79.41 %), the LMT model (79.41 %), and the RBF Neural Nets model (77.94 %). However, the MLP Neural Nets model has the lowest negative predictive value (76.47 %) indicating that the probability to correctly classify pixels to the non-landslide class is only 76.47 %. The highest one is the KLR model (88.24 %), followed by the SVM (85.29 %), the LMT, and the RBF Neural Nets (82.35 %).

The KLR model has the highest sensitivity (87.88 %) indicating that 87.88 % of the landslide pixels are correctly classified to the landslide class, followed by the SVM model (84.38 %), the RBF Neural Nets model (81.54 %), the LMT model (81.82 %), and the MLP Neural Nets model (80.25 %). The MLP Neural Nets model has the highest specificity (94.55 %) indicating that 94.55 % of the non-landslide pixels are correctly classified with respect to the non-landslide class. It is followed by the KLR model (85.71 %), the SVM model (80.56 %), the LMT model (80.00 %), and the RBF Neural Nets model (78.87 %).

Once the SVM, the MLP Neural Nets, the RBF Neural Nets, the KLR model, and the LMT models were successfully trained in the training process, they were used to calculate the landslide susceptibility indexes for all the pixels in the study area. Landslide susceptibility indexes were reclassified into five susceptibility levels: very high, high, moderate, low, and very low, using the equal area classification method (Pradhan and Lee 2010a, b).

### Model validation and comparison

Area under the ROC curves and Kappa index for the six landslide models on the validation dataset

Number | Landslide models | AUC | SE | 95 % CI | Kappa index |
---|---|---|---|---|---|

1 | SVM | 0.887 | 0.043 | 0.778–0.954 | 0.633 |

2 | MLP Neural Nets | 0.902 | 0.038 | 0.798–0.964 | 0.633 |

3 | RBF Neural Nets | 0.871 | 0.046 | 0.759–0.944 | 0.633 |

4 | KLR | 0.879 | 0.043 | 0.769–0.949 | 0.633 |

5 | LMT | 0.861 | 0.050 | 0.747–0.937 | 0.667 |

Model validation

Number | Parameter | SVM | MLP Neural Nets | RBF Neural Nets | KLR | LMT |
---|---|---|---|---|---|---|

1 | True positive | 24 | 26 | 23 | 25 | 24 |

2 | True negative | 25 | 23 | 26 | 24 | 26 |

3 | False positive | 6 | 4 | 7 | 5 | 6 |

4 | False negative | 5 | 7 | 4 | 6 | 4 |

5 | Positive predictive value (%) | 80.00 | 86.67 | 76.67 | 83.33 | 80.00 |

6 | Negative predictive value (%) | 83.33 | 76.67 | 86.67 | 80.00 | 86.67 |

7 | Sensitivity (%) | 82.76 | 78.79 | 85.19 | 80.65 | 85.71 |

8 | Specificity (%) | 80.65 | 85.19 | 78.79 | 82.76 | 81.25 |

The highest probability to correctly classify pixels to the landslide class is for the MLP Neural Nets model (86.67 %) whereas the RBF Neural Nets and the LMT models have the highest probability to correctly classify pixels to the non-landslide (Table 7). Sensitivity is the highest for the LMT model (85.71 %) indicating that 85.71 % of landslide pixels are correctly classified to the landslide. The MLP Neural Nets model shows the highest value of specificity (85.19 %) indicating that 85.19 % of non-landslide pixels are correctly classified to the non-landslide class.

*p*value is 0.000 less than 0.05 (Table 8) that satisfied the initial assumption; therefore, the null hypothesis is rejected. The average ranking of the five landslide susceptibility models for the study area is shown in Table 9.

Result of Friedman test for the five landslide susceptibility models with *α* = 0.05

Number | Value in | |
---|---|---|

1 | 42.28 | 1.46 × 10 |

Average ranking of the five landslide susceptibility models for the study area

Number | Landslide susceptibility models | Ranking |
---|---|---|

1 | SVM | 2.55 |

2 | MLP Neural Nets | 3.72 |

3 | RBF Neural Nets | 2.93 |

4 | KLR | 2.77 |

5 | LMT | 3.02 |

Since the result from the Friedman test shows only the significant differences of performance of the five landslide susceptibility models, the result is not able to provide comparisons between some of the five models. The Wilcoxon signed-rank test was therefore used to check the statistical significance of systematic pairwise differences between the landslide models. The null hypothesis is that there is no significant difference between the landslide susceptibility models at the significant level *α* = 5 %. The *p* value and *z* value are used to assess the significance of differences between the susceptibility models. When the *p* value is less than the significant level (0.05) and the *z* value exceeded the critical values of *z* (−1.96 and +1.96), the null hypothesis should be rejected and the performance of the susceptibility models is significantly different.

*p*values <0.05 and the

*z*values exceeded the critical values. The performance of the SVM and the LMT models is also significantly different (

*p*value = 0.013,

*z*value = −2.483). In contrast, the performance difference is not statistically significant for the five pairwise susceptibility models and is indicated with a symbol “*” in Table 10 (

*p*values >0.05 and

*z*values did not exceed the critical values).

Pairwise comparison for the five landslide susceptibility models using Wilcoxon signed-rank test (two-tailed)

Number | Pairwise comparison | | | Significance |
---|---|---|---|---|

1 | SVM vs. MLP Neural Nets | −5.676 | 0.000 | Yes |

2 | SVM vs. RBF Neural Nets* | −1.566 | 0.117 | No |

3 | SVM vs. KLR* | −1.871 | 0.061 | No |

4 | SVM vs. LMT | −2.483 | 0.013 | Yes |

5 | MLP Neural Nets vs. RBF Neural Nets | 3.582 | 0.000 | Yes |

6 | MLP Neural Nets vs. KLR | 6.536 | 0.000 | Yes |

7 | MLP Neural Nets vs. LMT | 4.801 | 0.000 | Yes |

8 | RBF Neural Nets vs. KLR* | 0.330 | 0.741 | No |

9 | RBF Neural Nets vs. LMT* | −0.391 | 0.696 | No |

10 | KLR vs. LMT* | −1.512 | 0.131 | No |

### Variable importance

The other conditioning factors tend to have different contributions depending on the types of models used. The LMT model yields small contributions by weathering (5.39 %) and relief amplitude (8.33 %) while these factors have much higher contribution to the other model. The SVM and the KLR models have the most relative balance in the contribution of the conditioning factors while the LMT and the MLP Neural Nets models showed a biggest difference in contribution of the conditioning factors.

## Discussion

Spatial prediction of landslides is considered to be one of the most difficult tasks in landslide hazard and risk assessment. Although various methods for shallow landslide susceptibility modeling has been proposed, the prediction accuracy of these methods is still debated (Akgun 2012). It is clear that the prediction accuracy of a landslide model depends on the method used. Some new machine learning methods such as SVM and J48 Decision tree have shown better results than conventional methods (Tien Bui et al. 2012a, 2013a). The ongoing development of the machine learning field and GIS has resulted in many new machine learning methods and techniques, i.e., KLR and LMT; therefore, the exploration of new methods and techniques for shallow landslide modeling is highly necessary. We addressed this issue in this paper with the evaluation and comparison of five machine learning methods, including some of the most popular methods (such as the MLP Neural Nets, the RBF Neural Nets, and the SVM) and some state-of-the art advanced machine learning methods such as the KLR and the LMT.

In general, the MLP Neural Nets and the KLR models yielded significant better results than the other models in terms of the overall classification accuracy; however, the KLR model has more balance in terms of positive and negative predictive values (Table 4). The overall classification accuracy of the other models is almost the same (from 80.15 to 82.35 %). The construction of the MLP Neural Nets model requires a large number of parameters, and it is not easy to determine the optimal values for these parameters (Pradhan and Lee 2010a, b). In this work, these parameters were chosen based on classification accuracy and area under the ROC curve (Fig. 5); therefore, it may guarantee that these are the optimal parameter values. In contrast, it is easier to obtain the best parameters for the other models.

The goodness-of-fit of the training data for the five susceptibility models are good; however, it differs between the models, although the KLR and the MLP Neural Nets models show the highest degree of fit with AUC values of the ROC being 0.941 for the KLR model and 0.934 for the MLP Neural Nets model. But using only AUC for the assessment of model performance may not be the best approach because in some cases a high value for the AUC may not be a guarantee for a high spatial accuracy of the models (Aguirre-Gutiérrez et al. 2013). Therefore, several statistical evaluation measures (Table 4) should be taken into account. Based on the probability that the model correctly classifies pixels to the landslide class in 95.59 % of the cases (Table 4), the MLP Neural Nets model has the highest degree of goodness-of-fit, closely followed by the KLR model. Regarding the prediction power of the five susceptibility models, the MLP Neural Nets model has the highest prediction capability in terms of AUC value (Table 6); the positive predictive value and other parameters are shown in Table 7. The prediction power of the MLP Neural Nets model differs statistically significantly with the other models (Tables 8 and 9).

Selection of landslide conditioning factors and the determination of their classes are key points that affect the quality of landslide susceptibility models (Costanzo et al. 2012; Irigaray et al. 2007) and have been discussed in Chacon et al. (2006). Although some methods have been proposed for the selection of conditioning factors such as linear correlation (Irigaray et al. 2007), Goodman–Kruskal and Kolmogorov–Smirnov test (Costanzo et al. 2012; Fernández et al. 2003), and GIS matrix combination method (Cross 2002), the standard guidelines for the selection of conditioning factors are still debated. The selection of conditioning factors is mainly carried out based on the analysis of the landslide types and the characteristics of the study area (Ayalew and Yamagishi 2005). In general, topography, geology, soil types, hydrology, geomorphology, and land use are widely accepted to use as conditioning factors in most landslide susceptibility modeling.

When building susceptibility models, it is logical to say that not all the selected conditioning factors have a good predictive ability, and in some cases, some of them create noises and reduce the prediction quality; therefore, the feature selection should be used (Martínez-Álvarez et al. 2013). In this study, no multicollinearity among the 12 conditioning factors was found (Tables 1 and 2), but the feature selection result showed that fault density, land use, TWI, and STI reveal non-predictive ability; therefore, the *Information Gain Ratio* was further considered for the conditioning factor selection.

Using the *Information Gain Ratio*, the amount of information that a conditioning factor can offer to the models is measured. The higher the value of the *Information Gain Ratio*, the higher the significance of the conditioning factor. In this study, the *Information Gain Ratio* is 0 for fault density, land use, TWI, and STI indicating that the four conditioning factors have null predictiveness to the models. This is because the distribution of landslide locations in the classes of the four conditioning factors is more even than the other conditioning factors. As a result, entropy of the two classes (landslide and non-landslide) did not decrease significantly when associated on the four conditioning factors.

The assessment of the contribution of the conditioning factors to the models is of great interest in the landslide analysis and has been discussed (Pavel et al. 2008). In general, slope is widely reported among the more effective instability factors (Costanzo et al. 2012; Pradhan and Lee 2010b; Van Den Eeckhaut et al. 2006). In this study, there are significant differences in the contribution of landslide conditioning factors to the landslide models, and this may lead to inconsistent conclusions. For example, relief amplitude contributes only 8.33 % for the LMT model but 14.2 % for the MLP Neural Nets model (Fig. 7). To reduce bias, the determination of important factors could be based on majority voting among many models (Xu et al. 2014). Accordingly, altitude, slope, and aspect are considered as the most important factors in this study. It is reasonable because most of the landslides in this study area occurred near the Da River and the lake of Son La hydropower where altitudes were less than 300 m within the slopes 8 to 35°. The west, south, and southwest are slope directions that are facing to the Da River and the lake of hydropower. The groundwater fluctuation gives rise to hydrodynamic forces that affect physical–mechanical properties of soils in these facing slopes (Do et al. 2000; Tuan and Dan 2012). According to Dan et al. (2011), the water that accumulates for the operation of the Son La hydropower has significantly increased and influenced the frequency of landslides in the basin, especially during the rainy season.

## Concluding remarks

This paper contributes to a systematic comparison and evaluation of five machine learning methods for landslide susceptibility modeling. The following issues are focused: (1) collinearity among the conditioning factors and feature selection using the *Information Gain Ratio*; (2) model assessment and comparison using the ROC, Kappa index, and the statistical evaluation measures; and (3) if the prediction capability of the landslide models is significantly different. According to this case study, the *Information Gain Ratio* could be used for the feature selection, and the MLP Neural Nets model and the KLR model yield more accurate and reliable results than the other models. The LMT model with a prediction capability of 86.1 % and a positive predictive value of 80 % is being considered as a promising technique for landslide susceptibility mapping. As a final conclusion, the results from this study may be useful for decision making and policy planning in areas prone to landslides.

## Notes

### Acknowledgment

This research was supported by the Geographic Information System group, Department of Business Administration and Computer Science, Faculty of Art and Sciences, Telemark University College, Bø i Telemark, Norway. The authors would like to thank Professor Candan Gokceoglu and three anonymous reviewers for their valuable and constructive comments on the earlier version of the manuscript.

## References

- Abe S (2010) Support vector machines for pattern classification. Springer, LondonCrossRefGoogle Scholar
- Aguirre-Gutiérrez J, Carvalheiro LG, Polce C, van Loon EE, Raes N, Reemer M, Biesmeijer JC (2013) Fit-for-purpose: species distribution model performance depends on evaluation criteria—Dutch hoverflies as a case study. PLoS One 8:e63708CrossRefGoogle Scholar
- Akgun A (2012) A comparison of landslide susceptibility maps produced by logistic regression, multi-criteria decision, and likelihood ratio methods: a case study at İzmir, Turkey. Landslides 9:93–106CrossRefGoogle Scholar
- Akgun A, Sezer EA, Nefeslioglu HA, Gokceoglu C, Pradhan B (2012) An easy-to-use MATLAB program (MamLand) for the assessment of landslide susceptibility using a Mamdani fuzzy algorithm. Comput Geosci 38:23–34CrossRefGoogle Scholar
- Allison PD (1999) Logistic regression using the SAS system: theory and application. SAS Institute, Inc., CaryGoogle Scholar
- Atkinson PM, Massari R (1998) Generalised linear modelling of susceptibility to landsliding in the central Apennines, Italy. Comput Geosci 24:373–385CrossRefGoogle Scholar
- Ayalew L, Yamagishi H (2005) The application of GIS-based logistic regression for landslide susceptibility mapping in the Kakuda-Yahiko mountains, central Japan. Geomorphology 65:15–31CrossRefGoogle Scholar
- Ballabio C, Sterlacchini S (2012) Support vector machines for landslide susceptibility mapping: the Staffora River Basin case study, Italy. Math Geosci 44:47–70CrossRefGoogle Scholar
- Beasley TM, Zumbo BD (2003) Comparison of aligned Friedman rank and parametric methods for testing interactions in split-plot designs. Comput Stat Data Anal 42:569–593CrossRefGoogle Scholar
- Belsley D (1991) A guide to using the collinearity diagnostics. Comput Sci Econ Manag 4:33–50Google Scholar
- Booth GD, Niccolucci MJ, Schuster EG (1994) Identifying proxy sets in multiple linear regression: an aid to better coefficient interpretation. US Dept of Agriculture Forest Service, OgdenGoogle Scholar
- Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30:1145–1159CrossRefGoogle Scholar
- Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Wadsworth, BelmontGoogle Scholar
- Brenning A (2005) Spatial prediction models for landslide hazards: review, comparison and evaluation. Nat Hazards Earth Syst Sci 5:853–862CrossRefGoogle Scholar
- Carrara A, Pike RJ (2008) GIS technology and models for assessing landslide hazard and risk. Geomorphology 94:257–260CrossRefGoogle Scholar
- Cawley G, Talbot NC (2008) Efficient approximate leave-one-out cross-validation for kernel logistic regression. Mach Learn 71:243–264CrossRefGoogle Scholar
- Chacon J, Irigaray C, Fernandez T, El Hamdouni R (2006) Engineering geology maps: landslides and geographical information systems. Bull Eng Geol Environ 65:341–411CrossRefGoogle Scholar
- Chung C-J, Fabbri AG (2008) Predicting landslides for risk analysis—spatial models tested by a cross-validation technique. Geomorphology 94:438–452CrossRefGoogle Scholar
- Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRefGoogle Scholar
- Costanzo D, Rotigliano E, Irigaray C, Jiménez-Perálvarez JD, Chacón J (2012) Factors selection in landslide susceptibility modelling on large scale following the GIS matrix method: application to the River Beiro Basin (Spain). Nat Hazards Earth Syst Sci 12:327–340CrossRefGoogle Scholar
- Costanzo D, Chacón J, Conoscenti C, Irigaray C, Rotigliano E (2014) Forward logistic regression for earth-flow landslide susceptibility assessment in the Platani river basin (southern Sicily, Italy). Landslides 11:639–653CrossRefGoogle Scholar
- Cross M (2002) Landslide susceptibility mapping using the matrix assessment approach: a Derbyshire case study. In: Griffiths JS (ed) Mapping in engineering geology, The Geological Society. Key Issue in Earth Sciences, London, pp 247–261Google Scholar
- Dan NT, Tuan TA, Thu TH, Hong PV, Hung LQ, Luong NV, Hai NT, Nhung H, Ha NTV, Thu DH, Thanh LV, Hien D, Mai D (2011) Application of remote sensing, GIS, and GPS for the study of landslides at the Son La hydropower basin and proposed remedial measures. Institute of Marine Geology & Geophysics, Vietnam Academy of Science and Technology, Hanoi, p 140Google Scholar
- D’Arco M, Liccardo A, Pasquino N (2012) ANOVA-based approach for DAC diagnostics. IEEE Trans Instrum Meas 61:1874–1882CrossRefGoogle Scholar
- Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evol Comput 1:3–18CrossRefGoogle Scholar
- Do T, Bui Minh T, Truong Minh T, Trinh Xuan H, Nguyen Phuong M (2000) The investigation and assessment of environmental geology for the Son La hydropower basin and its surrounding areas. Ministry of Science, Technology and Environment of Vietnam, Hanoi, p 231Google Scholar
- Doetsch P, Buck C, Golik P, Hoppe N, Kramp M, Laudenberg J, Oberdörfer C, Steingrube P, Forster J and Mauser A (2009) Logistic model trees with AUC split criterion for the KDD Cup 2009 Small ChallengeGoogle Scholar
- Dormann CF, Elith J, Bacher S, Buchmann C, Carl G, Carré G, Marquéz JRG, Gruber B, Lafourcade B, Leitão PJ, Münkemüller T, McClean C, Osborne PE, Reineking B, Schröder B, Skidmore AK, Zurell D, Lautenbach S (2013) Collinearity: a review of methods to deal with it and a simulation study evaluating their performance. Ecography 36:27–46CrossRefGoogle Scholar
- Dovzhikov AE, Mi BP, Vasilevskaya ED, Zhamoida AI, Ivanov GV, Izokh EP, Huu LD, Mareichev AM, Tien NV, Tri NT, Luong TD, Kuang PV, Long PD (1965) Geology of northern Vietnam. Science and Technology, Hanoi, p 668Google Scholar
- Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17:191–209CrossRefGoogle Scholar
- Duc D (2013) Rainfall-triggered large landslides on 15 December 2005 in Van Canh district, Binh Dinh province, Vietnam. Landslides 10:219–230CrossRefGoogle Scholar
- Ercanoglu M, Gokceoglu C (2002) Assessment of landslide susceptibility for a landslide-prone area (north of Yenice, NW Turkey) by fuzzy approach. Environ Geol 41:720–730CrossRefGoogle Scholar
- Erener A, Düzgün H (2010) Improvement of statistical landslide susceptibility mapping by using spatial and global regression methods in the case of More and Romsdal (Norway). Landslides 7:55–68CrossRefGoogle Scholar
- Felicisimo A, Cuartero A, Remondo J, Quiros E (2013) Mapping landslide susceptibility with logistic regression, multiple adaptive regression splines, classification and regression trees, and maximum entropy methods: a comparative study. Landslides 10:175–189CrossRefGoogle Scholar
- Fernández T, Irigaray C, El Hamdouni R, Chacón J (2003) Methodology for landslide susceptibility mapping by means of a GIS. Application to the Contraviesa Area (Granada, Spain). Nat Hazards 30:297–308CrossRefGoogle Scholar
- Forest Inventory and Planning Institute (2005) The forest map of Vietnam scale 1:50 000. Vietnam Forest Inventory and Planning Institute, HanoiGoogle Scholar
- Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:675–701CrossRefGoogle Scholar
- Gama J (2004) Functional trees. Mach Learn 55:219–250CrossRefGoogle Scholar
- General Department of Geology and Minerals of Vietnam (2000) Geological and mineral resources maps scale of 1:200,000Google Scholar
- Gil D and Johnsson M (2010) Supervised SOM based architecture versus multilayer perceptron and RBF networks. Proc Linköping Electron Conf: 15-24Google Scholar
- Gokceoglu C, Aksoy H (1996) Landslide susceptibility mapping of the slopes in the residual soils of the Mengen region (Turkey) by deterministic stability analyses and image processing techniques. Eng Geol 44:147–161CrossRefGoogle Scholar
- Gokceoglu C, Sezer E (2009) A statistical assessment on international landslide literature (1945–2008). Landslides 6:345–351CrossRefGoogle Scholar
- Gomez H, Kavzoglu T (2005) Assessment of shallow landslide susceptibility using artificial neural networks in Jabonosa River Basin, Venezuela. Eng Geol 78:11–27CrossRefGoogle Scholar
- Guzzetti F, Galli M, Reichenbach P, Ardizzone F, Cardinali M (2006a) Landslide hazard assessment in the Collazzone area, Umbria, central Italy. Nat Hazards Earth Syst Sci 6:115–131CrossRefGoogle Scholar
- Guzzetti F, Reichenbach P, Ardizzone F, Cardinali M, Galli M (2006b) Estimating the quality of landslide susceptibility models. Geomorphology 81:166–184CrossRefGoogle Scholar
- Hair JF, Black WC, Babin BJ, Anderson RE (2009) Multivariate data analysis. Prentice Hall, New YorkGoogle Scholar
- Haykin S (1998) Neural networks: a comprehensive foundation (2nd edition). Prentice Hall, Upper Saddle RiverGoogle Scholar
- Hungr O, Fell R, Couture R, Eberhardt E (2005) Landslide risk management. CRC PressGoogle Scholar
- Hunter E, Matin J, Stone P (1966) Experiments in induction. Academic, New YorkGoogle Scholar
- Irigaray C, Fernández T, El Hamdouni R, Chacón J (2007) Evaluation and validation of landslide-susceptibility maps obtained by a GIS matrix method: examples from the Betic Cordillera (southern Spain). Nat Hazards 41:61–79CrossRefGoogle Scholar
- Jebur MN, Pradhan B, Tehrany MS (2014) Optimization of landslide conditioning factors using very high-resolution airborne laser scanning (LiDAR) data at catchment scale. Remote Sens Environ 152:150–165CrossRefGoogle Scholar
- Jia N, Mitani Y, Xie MW, Djamaluddin I (2012) Shallow landslide hazard assessment using a three-dimensional deterministic model in a mountainous area. Comput Geotech 45:1–10CrossRefGoogle Scholar
- Jiménez-Perálvarez JD, Irigaray C, El Hamdouni R, Chacón J (2011) Landslide susceptibility mapping in a semi-arid mountain environment: an example from the southern slopes of Sierra Nevada (Granada, Spain). Bull Eng Geol Environ 70:265–277CrossRefGoogle Scholar
- Kavzoglu T, Colkesen I (2009) A kernel functions analysis for support vector machines for land cover classification. Int J Appl Earth Obs Geoinform 11:352–359CrossRefGoogle Scholar
- Kavzoglu T, Mather PM (2003) The use of backpropagating artificial neural networks in land cover classification. Int J Remote Sens 24:4907–4938CrossRefGoogle Scholar
- Kavzoglu T, Kutlug Sahin E, Colkesen I (2014a) An assessment of multivariate and bivariate approaches in landslide susceptibility mapping: a case study of Duzkoy district. Nat Hazards 1-26Google Scholar
- Kavzoglu T, Sahin E, Colkesen I (2014b) Landslide susceptibility mapping using GIS-based multi-criteria decision analysis, support vector machines, and logistic regression. Landslides 11:425–439CrossRefGoogle Scholar
- Keith TZ (2006) Multiple regressions and beyond. Pearson, BostonGoogle Scholar
- Kononenko I (1994) Estimating attributes: analysis and extensions of relief. In: Bergadano F, Raedt L (eds) Machine learning: ECML-94. Springer, Berlin, pp 171–182CrossRefGoogle Scholar
- Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRefGoogle Scholar
- Landwehr N, Hall M, Frank E (2005) Logistic model trees. Mach Learn 59:161–205CrossRefGoogle Scholar
- Lee S (2005) Application of logistic regression model and its validation for landslide susceptibility mapping using GIS and remote sensing data journals. Int J Remote Sens 26:1477–1491CrossRefGoogle Scholar
- Lee S (2007) Application and verification of fuzzy algebraic operators to landslide susceptibility mapping. Environ Geol 52:615–623CrossRefGoogle Scholar
- Lee S, Ryu JH, Min KD, Won JS (2003) Landslide susceptibility analysis using GIS and artificial neural network. Earth Surf Process Landf 28:1361–1376CrossRefGoogle Scholar
- Liao D, Valliant R (2012) Variance inflation factors in the analysis of complex survey data. Surv Methodol 38:53–62Google Scholar
- Martínez-Álvarez F, Reyes J, Morales-Esteban A, Rubio-Escudero C (2013) Determining the best set of seismicity indicators to predict earthquakes. Two case studies: Chile and the Iberian Peninsula. Knowl-Based Syst 50:198–210CrossRefGoogle Scholar
- Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philos Trans R Soc Lond Ser A, Containing Pap Math Phys Charact 209:415–446CrossRefGoogle Scholar
- Montgomery DR, Dietrich WE (1994) A physically-based model for the topographic control on shallow landsliding. Water Resour Res 30:1153–1171CrossRefGoogle Scholar
- Nefeslioglu HA, Sezer E, Gokceoglu C, Bozkir AS, Duman TY (2010) Assessment of landslide susceptibility by decision trees in the metropolitan area of Istanbul, Turkey. Math Probl EngGoogle Scholar
- Pavel M, Fannin RJ, Nelson JD (2008) Replication of a terrain stability mapping using an artificial neural network. Geomorphology 97:356–373CrossRefGoogle Scholar
- Pourghasemi H, Pradhan B, Gokceoglu C (2012a) Application of fuzzy logic and analytical hierarchy process (AHP) to landslide susceptibility mapping at Haraz watershed, Iran. Nat Hazards 63:965–996CrossRefGoogle Scholar
- Pourghasemi H, Pradhan B, Gokceoglu C, Moezzi KD (2012b) A comparative assessment of prediction capabilities of Dempster–Shafer and weights-of-evidence models in landslide susceptibility mapping using GIS. Geomatics Nat Hazards Risk 4:93–118CrossRefGoogle Scholar
- Pradhan B (2011) Manifestation of an advanced fuzzy logic model coupled with geo-information techniques to landslide susceptibility mapping and their comparison with logistic regression modelling. Environ Ecol Stat 18:471–493CrossRefGoogle Scholar
- Pradhan B (2012) A comparative study on the predictive ability of the decision tree, support vector machine and neuro-fuzzy models in landslide susceptibility mapping using GIS. Comput Geosci 51:350–365CrossRefGoogle Scholar
- Pradhan B, Lee S (2010a) Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modelling. Environ Model Softw 25:747–759CrossRefGoogle Scholar
- Pradhan B, Lee S (2010b) Regional landslide susceptibility analysis using back-propagation neural network model at Cameron Highland, Malaysia. Landslides 7:13–30CrossRefGoogle Scholar
- Pradhan B, Lee S, Buchroithner MF (2010a) A GIS-based back-propagation neural network model and its cross-application and validation for landslide susceptibility analyses. Comput Environ Urban Syst 34:216–235CrossRefGoogle Scholar
- Pradhan B, Sezer EA, Gokceoglu C, Buchroithner MF (2010b) Landslide susceptibility mapping by neuro-fuzzy approach in a landslide-prone area (Cameron Highlands, Malaysia). IEEE Trans Geosci Remote Sens 48:4164–4177CrossRefGoogle Scholar
- Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San Mateo, CA, USAGoogle Scholar
- Saito H, Nakayama D, Matsuyama H (2009) Comparison of landslide susceptibility based on a decision-tree model and actual landslide occurrence: the Akaishi mountains, Japan. Geomorphology 109:108–121CrossRefGoogle Scholar
- Sasikala S, AppavualiasBalamurugan S, Geetha S (2014) Multi filtration feature selection (MFFS) to improve discriminatory ability in clinical data set. Appl Comput Inform. doi: 10.1016/j.aci.2014.03.002 Google Scholar
- Schuerman J (1983) Principal components analysis. Multivariate analysis in the human services. Springer, Netherlands, pp 93–119Google Scholar
- Şenkal O, Kuleli T (2009) Estimation of solar radiation over turkey using artificial neural network and satellite data. Appl Energy 86:1222–1228CrossRefGoogle Scholar
- Sezer EA, Pradhan B, Gokceoglu C (2011) Manifestation of an adaptive neuro-fuzzy model on landslide susceptibility mapping: Klang valley, Malaysia. Expert Syst Appl 38:8208–8219CrossRefGoogle Scholar
- Sossa H, Guevara E (2014) Efficient training for dendrite morphological neural networks. Neurocomputing 131:132–142CrossRefGoogle Scholar
- Thanh L, De Smedt F (2014) Slope stability analysis using a physically based model: a case study from a Luoi district in Thua Thien-Hue province, Vietnam. Landslides 11:897–907CrossRefGoogle Scholar
- Tien Bui D (2012) Modeling of rainfall-induced landslide hazard for the hoa binh province of vietnam. Norwegian University of Life Sciences. Ph.D Thesis, 192pGoogle Scholar
- Tien Bui D, Lofman O, Revhaug I, Dick O (2011) Landslide susceptibility analysis in the Hoa Binh province of Vietnam using statistical index and logistic regression. Nat Hazards 59:1413–1444CrossRefGoogle Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I (2012a) Landslide susceptibility assessment in Vietnam using support vector machines, decision tree and naïve Bayes models. Math Probl Eng 2012:1–26CrossRefGoogle Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I, Dick OB (2012b) Application of support vector machines in landslide susceptibility assessment for the Hoa Binh province (Vietnam) with kernel functions analysis. In: Seppelt R, Voinov AA, Lange S, Bankamp D (eds) Proceedings of the iEMSs Sixth Biennial Meeting: International Congress on Environmental Modelling and Software (iEMSs 2012) International Environmental Modelling and Software Society, Leipzig, Germany, July 2012Google Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I, Dick OB (2012c) Landslide susceptibility assessment in the Hoa Binh province of Vietnam: a comparison of the Levenberg-Marquardt and Bayesian regularized neural networks. Geomorphology 171–172:12–29CrossRefGoogle Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I, Dick OB (2012d) Landslide susceptibility mapping at Hoa Binh province (Vietnam) using an adaptive neuro-fuzzy inference system and GIS. Comput Geosci 45:199–211CrossRefGoogle Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I and Dick OB (2012e) Spatial prediction of landslide hazards in hoa binh province (vietnam): a comparative assessment of the efficacy of evidential belief functions and fuzzy logic models. Catena 96: 28-40Google Scholar
- Tien Bui D, Ho TC, Revhaug I, Pradhan B, Nguyen D (2013a) Landslide susceptibility mapping along the national road 32 of Vietnam using GIS-based j48 decision tree classifier and its ensembles. In: Buchroithner M, Prechtel N, Burghardt D (eds) Cartography from pole to pole. Springer, Berlin, pp 303–317Google Scholar
- Tien Bui D, Pradhan B, Lofman O, Revhaug I, Dick O (2013b) Regional prediction of landslide hazard using probability analysis of intense rainfall in the Hoa Binh province, Vietnam. Nat Hazards 66:707–730CrossRefGoogle Scholar
- Tien Bui D, Pradhan B, Revhaug I, Trung Tran C (2014) A comparative assessment between the application of fuzzy unordered rules induction algorithm and j48 decision tree models in spatial prediction of shallow landslides at Lang Son city, Vietnam. In: Srivastava PK, Mukherjee S, Gupta M, Islam T (eds) Remote sensing applications in environmental research. Springer International Publishing, pp 87–111Google Scholar
- Tuan TA, Dan NT (2012) Landslide susceptibility mapping and zoning in the Son La hydropower catchment area using the analytical hierarchy process. J Sci Earth (Vietnamese): 223–232Google Scholar
- Tunusluoglu MC, Gokceoglu C, Nefeslioglu HA, Sonmez H (2008) Extraction of potential debris source areas by logistic regression technique: a case study from Barla, Besparmak and Kapi mountains (NW Taurids, Turkey). Environ Geol 54:9–22CrossRefGoogle Scholar
- Van Den Eeckhaut M, Vanwalleghem T, Poesen J, Govers G, Verstraeten G, Vandekerckhove L (2006) Prediction of landslide susceptibility using rare events logistic regression: a case-study in the Flemish Ardennes (Belgium). Geomorphology 76:392–410CrossRefGoogle Scholar
- Van Den Eeckhaut M, Reichenbach P, Guzzetti F, Rossi M, Poesen J (2009) Combined landslide inventory and susceptibility assessment based on different mapping units: an example from the Flemish Ardennes, Belgium. Nat Hazards Earth Syst Sci 9:507–521CrossRefGoogle Scholar
- Van Westen CJ, Terlien MTJ (1996) An approach towards deterministic landslide hazard analysis in GIS. A case study from Manizales (Colombia). Earth Surf Process Landf 21:853–868CrossRefGoogle Scholar
- Van TT, Anh DT, Hieu HH, Giap NX, Ke TD, Nam TD, Ngoc D, Ngoc DTY, Thai TN, Thang DV, Tinh NV, Tuat LT, Tung NT, Tuy PK, Viet HA (2006) Investigation and assessment of the current status and potential of landslides in some sections of the Ho Chi Minh Road, National Road 1a and proposed remedial measures to prevent landslides from threat of safety of people, property, and infrastructure. Vietnam Institute of Geosciences and Mineral Resources, Hanoi, p 249Google Scholar
- Vapnik VN (1998) Statistical learning theory. Wiley-InterscienceGoogle Scholar
- Vorpahl P, Elsenbeer H, Marker M, Schroder B (2012) How can statistical models help to determine driving factors of landslides? Ecol Model 239:27–39CrossRefGoogle Scholar
- Walter SD (2002) Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Stat Med 21:1237–1256CrossRefGoogle Scholar
- Witten IH, Frank E, Mark AH (2011) Data mining: practical machine learning tools and techniques (third edition). Morgan Kaufmann, BurlingtonGoogle Scholar
- Wu W, Sidle RC (1995) A distributed slope stability model for steep forested basins. Water Resour Res 31:2097–2110CrossRefGoogle Scholar
- Xiaomeng W, Borgelt C (2004) Information measures in fuzzy decision trees. Fuzzy Systems, 2004 Proceedings 2004 I.E. International Conference on, pp 85–90 vol. 81Google Scholar
- Xu L, Li J, Brenning A (2014) A comparative study of different classification techniques for marine oil spill identification using RADARSAT-1 imagery. Remote Sens Environ 141:14–23CrossRefGoogle Scholar
- Yao X, Tham LG, Dai FC (2008) Landslide susceptibility mapping based on support vector machine: a case study on natural slopes of Hong Kong, China. Geomorphology 101:572–582CrossRefGoogle Scholar
- Yem NT (2006) Assessment of landslides, flash floods, and debris flows in selected prone areas in the northern mountainous Vietnam and recommendation of remedial measures to prevent and mitigate potential damages. pp 166Google Scholar
- Yilmaz I (2009) Landslide susceptibility mapping using frequency ratio, logistic regression, artificial neural networks and their comparison: a case study from Kat landslides (Tokat—Turkey). Comput Geosci 35:1125–1138CrossRefGoogle Scholar
- Yilmaz I (2010) Comparison of landslide susceptibility mapping methodologies for Koyulhisar, Turkey: conditional probability, logistic regression, artificial neural networks, and support vector machine. Environ Earth Sci 61:821–836CrossRefGoogle Scholar
- Zhuang L, Dai HH (2006) Parameter optimization of kernel-based one-class classifier on imbalance text learning. PRICAI 2006. Trends Artif Intell Proc 4099:434–443Google Scholar