Skip to main content

Advertisement

SpringerLink
Log in

Liquefaction prediction with robust machine learning algorithms (SVM, RF, and XGBoost) supported by genetic algorithm-based feature selection and parameter optimization from the perspective of data processing

  • Original Article
  • Published:
Environmental Earth Sciences Aims and scope Submit manuscript

Abstract

Liquefaction prediction is an important issue in the seismic design of engineering structures, and research on this topic has been continuing in current literature using different methods, including experimental, numerical, or soft computing. In this paper, three robust machine learning (ML) algorithms are applied to predict soil liquefaction using a set of 411 shear wave velocity case records. The Genetic Algorithm (GA) based feature selection (FS) and parameter optimization of Random Forest (RF), Support Vector Machines (SVM), and eXtreme Gradient Boosting (XGBoost) algorithms are utilized to improve the accuracy of the liquefaction prediction models. Simple Random Sampling (SRS) and Stratified Random Sampling (StrRS) are used for data sampling, and also SMOTE algorithm are applied to prepare the balanced training sets. The results of robust ML algorithms are assessed based on well-known five performance matrices, namely Accuracy (Acc), Kappa, Precision, Recall, and F-Measure. Evaluation of the results is made separately for each ML algorithm considering sampling data generated from SRS, StrRS, and SMOTE. As a result, the XGBoost model is more accurate (Acc = 96%) than RF (Acc = 93%) and SVM (Acc = 91%) in the case of the SMOTE algorithm. This study reveals the superiority of the XGBoost algorithm in the liquefaction prediction and shows how the accuracy measures tend to improve when the predictive models are trained using balanced samples with StrRS and SMOTE sampling strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 3

Similar content being viewed by others

References

  • Acharya AS, Prakash A, Saxena P, Nigam A (2013) Sampling: why and how of it. Indian J Med SpEcialties 4(2):330–333

    Google Scholar 

  • Al Imran A, Rifat RI, Mohammad R (2020). Enhancing the classification performance of lower back pain symptoms using genetic algorithm-based feature selection. In: Proceedings of International Joint Conference on Computational Intelligence (pp. 455–469). Springer, Singapore.

  • Andrus RD, Stokoe KH II (2000) Liquefaction resistance of soils from shear-wave velocity. J Geotech Geoenviron Eng 126(11):1015–1025

    Article  Google Scholar 

  • Aziz ASA, Azar AT, Salama MA, Hassanien AE, Hanafy SE-O (2013) Genetic algorithm with different feature selection techniques for anomaly detectors generation, Federated Conference on Computer Science and Information Systems. IEEE, pp. 769–774.

  • Ben-David A (2007) A lot of randomness is hiding in accuracy. Eng App Artif Intell 20(7):875–885

    Article  Google Scholar 

  • Bian J, Peng XG, Wang Y, Zhang H (2016). An efficient cost-sensitive feature selection using chaos genetic algorithm for class imbalance problem. Math Probl Eng, 2016.

  • Boulanger R, Idriss I (2014) CPT and SPT based liquefaction triggering procedures. Report No. UCD/CGM.-14. 1.

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  • Breiman L (2018) randomForest: Breiman and Cutler’s random forests for classification and regression. R package version 4.6–12. Software available at URL: https://cran.r-project.org/package= randomForest. 0098–5589.

  • Brownlee J (2020) Imbalanced classification with Python: better metrics, balance skewed classes, cost-sensitive learning. Machine Learning Mastery, 463pp.

  • Cao Z, Youd TL, Yuan X (2011) Gravelly soils that liquefied during 2008 Wenchuan China earthquake, Ms=8.0. Soil Dyn Earthq Eng 31(8):1132–1143

    Article  Google Scholar 

  • Chawla NV (2009) Data mining for imbalanced datasets: an overview. Data mining and knowledge discovery handbook, pp 875–886.

  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  • Chen H, Jiang W, Li C, Li R (2013) A heuristic feature selection approach for text categorization by using chaos optimization and genetic algorithm. Math Probl Eng. https://doi.org/10.1155/2013/524017

    Article  Google Scholar 

  • Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794.

  • Chen T, He T, Benesty M, Khotilovich V, Tang Y, Cho H (2020) Xgboost: extreme gradient boosting. R package version 1.1.1.1. 1(4), 1–4.

  • Chu DB, Stewart JP, Youd TL, Chu B (2006) Liquefaction-induced lateral spreading in near-fault regions during the 1999 Chi-Chi Taiwan Earthquake. J Geotech Geoenviron Eng 132(12):1549–1565

    Article  Google Scholar 

  • Cohen J (1960) A coefficient of agreement for nominal scales. Educ PSychol Meas 20(1):37–46

    Article  Google Scholar 

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  Google Scholar 

  • Das SK, Mohanty R, Mohanty M, Mahamaya M (2020) Multi-objective feature selection (MOFS) algorithms for prediction of liquefaction susceptibility of soil based on in situ test methods. Nat Hazards 103(2):2371–2393

    Article  Google Scholar 

  • Demir S, Sahin EK (2022) Comparison of tree-based machine learning algorithms for predicting liquefaction potential using canonical correlation forest, rotation forest, and random forest based on CPT data. Soil Dynam Earthq Eng 154:107130. https://doi.org/10.1016/j.soildyn.2021.107130

    Article  Google Scholar 

  • Elgeldawi E, Sayed A, Galal AR, Zaki AM (2021) Hyperparameter tuning for machine learning algorithms used for Arabic sentiment analysis. Informatics MDPI 8(4):79

    Article  Google Scholar 

  • Fernández A, García S, Galar M, Prati RC, Krawczyk B, Herrera F (2018) Learning from imbalanced data sets. Springer

    Book  Google Scholar 

  • Fröhlich H, Chapelle O, Schölkopf B (2002) Feature selection for support vector machines by means of genetic algorithms. In: Proceeding ICTAI.

  • Gen M, Cheng R, Lin L (2008) Network models and optimization: Multiobjective genetic algorithm approach. Springer Science & Business Media, NY

    Google Scholar 

  • Goh AT (1994) Seismic liquefaction potential assessed by neural networks. J Geotech Eng 120(9):1467–1480

    Article  Google Scholar 

  • Goh AT (1996) Neural-network modeling of CPT seismic liquefaction data. J Geotech Eng 122(1):70–73

    Article  Google Scholar 

  • Guyon I, Saffari A, Dror G, Cawley G (2010) Model selection: beyond the bayesian/frequentist divide. J Mach Learn Res 11(1):61–87

    Google Scholar 

  • Hanna AM, Ural D, Saygili G (2007) Neural network model for liquefaction potential in soil deposits using Turkey and Taiwan earthquake data. Soil Dynam Earthq Eng 27(6):521–540

    Article  Google Scholar 

  • Hastie T, Tibshirani R, Friedman J (2009) Random forests, the elements of statistical learning. Springer, pp. 587–604.

  • He H, Ma Y (2013) Imbalanced learning: foundations, algorithms, and applications. John Wiley & Sons Inc, Hoboken, New Jersey

    Book  Google Scholar 

  • Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844

    Article  Google Scholar 

  • Holland J (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, p 183

    Google Scholar 

  • Ishihara K, Yasuda S, Nagase H (1996) Soil characteristics and ground damage. Soils Found 36(1):109–118

    Article  Google Scholar 

  • James G, Witten D, Hastie T, Tibshirani R (2013) An introduction to statistical learning with applications in R. Springer, New York

    Book  Google Scholar 

  • Japkowicz N, Shah M (2011) Evaluating learning algorithms: a classification perspective. Cambridge University Press, New York

    Book  Google Scholar 

  • Kardani N, Zhou A, Nazem M, Shen S-L (2020) Estimation of bearing capacity of piles in cohesionless soil using optimised machine learning approaches. Geotech Geol Eng 38(2):2271–2291

    Article  Google Scholar 

  • Kardani N, Zhou A, Nazem M, Shen S-L (2021) Improved prediction of slope stability using a hybrid stacking ensemble method based on finite element analysis and field data. J Rock Mech Geotech Eng 13(1):188–201

    Article  Google Scholar 

  • Kayen R, Moss R, Thompson E, Seed R, Cetin K, Kiureghian AD, Tanaka Y, Tokimatsu K (2013) Shear-wave velocity–based probabilistic and deterministic assessment of seismic soil liquefaction potential. J Geotech Geoenviron Eng 139(3):407–419

    Article  Google Scholar 

  • Keyport RN, Oommen T, Martha TR, Sajinkumar K, Gierke JS (2018) A comparative analysis of pixel-and object-based detection of landslides from very high-resolution images. Int J Appl Earth Obs Geoinf 64:1–11

    Google Scholar 

  • Khanam Z, Alkhaldi S (2019). An intelligent recommendation engine for selecting the University for Graduate Courses in KSA: SARS Student Admission Recommender System. In: International Conference on Inventive Computation Technologies (pp. 711–722). Springer, Cham.

  • Kramer O (2017) Genetic algorithms, genetic algorithm essentials. Springer, New York, pp 11–19

    Google Scholar 

  • Kramer SL (2018) Past, present, and future developments in liquefaction hazard analysis, Developments in earthquake geotechnics. Springer, New York, pp 51–60

    Book  Google Scholar 

  • Kuhn M, Johnson K (2013) Applied predictive modeling. Springer, NY

    Book  Google Scholar 

  • Kurnaz TF, Kaya Y (2019) A novel ensemble model based on GMDH-type neural network for the prediction of CPT-based soil liquefaction. Environ Earth Sci 78(11):1–14

    Article  Google Scholar 

  • Le T-T, Asteris PG, Lemonis ME (2021) Prediction of axial load capacity of rectangular concrete-filled steel tube columns using machine learning techniques. Eng Comput. https://doi.org/10.1007/s00366-021-01461-0

    Article  Google Scholar 

  • Li H, Yuan D, Ma X, Cui D, Cao L (2017) Genetic algorithm for the optimization of features and neural networks in ECG signals classification. Sci Rep 7(1):1–12

    Google Scholar 

  • Li W (2004) Using genetic algorithm for network intrusion detection. In: Proceedings of the United States department of energy cyber security group. 1, 1–8.

  • Lin S, Zheng H, Han B, Li Y, Han C, Li W (2022) Comparative performance of eight ensemble learning approaches for the development of models of slope stability prediction. Acta Geotech. https://doi.org/10.1007/s11440-021-01440-1

    Article  Google Scholar 

  • Ma M, Zhao G, He B, Li Q, Dong H, Wang S, Wang Z (2021) XGBoost-based method for flash flood risk assessment. J Hydrol 598:126382

    Article  Google Scholar 

  • Marsland S (2015) Machine learning: an algorithmic perspective. CRC Press, FL

    Google Scholar 

  • Maxwell AE, Warner TA, Fang F (2018) Implementation of machine-learning classification in remote sensing: an applied review. Int J Remote Sens 39(9):2784–2817

    Article  Google Scholar 

  • Meyer D, Wien FT (2021) Support vector machines. The Interface to libsvm in package e1071.

  • Mittal M, Satapathy SC, Pal V, Agarwal B, Goyal LM, Parwekar P (2021) Prediction of coefficient of consolidation in soil using machine learning techniques. MicroproceSs Microsyst 82:103830

    Article  Google Scholar 

  • Nejad AS, Güler E, Özturan M (2018) Evaluation of liquefaction potential using random forest method and shear wave velocity results, 2018 International Conference on Applied Mathematics & Computational Science (ICAMCS. NET). IEEE, pp. 23–233.

  • Njock PGA, Shen S-L, Zhou A, Lyu H-M (2020) Evaluation of soil liquefaction using AI technology incorporating a coupled ENN/t-SNE model. Soil Dynam Earthq Eng 130:105988

    Article  Google Scholar 

  • Nwanganga F, Chapple M (2020) Practical machine learning in R. John Wiley & Sons, USA

    Book  Google Scholar 

  • Pham BT, Nguyen-Thoi T, Ly H-B, Nguyen MD, Al-Ansari N, Tran V-Q, Le T-T (2020) Extreme learning machine based prediction of soil shear strength: a sensitivity analysis using Monte Carlo simulations and feature backward elimination. Sustainability 12(6):2339

    Article  Google Scholar 

  • Pham BT, Nguyen MD, Bui K-TT, Prakash I, Chapi K, Bui DT (2019) A novel artificial intelligence approach based on multi-layer perceptron neural network and biogeography-based optimization for predicting coefficient of consolidation of soil. CATENA 173:302–311

    Article  Google Scholar 

  • Pisner DA, Schnyer DM (2020) Chapter 6—Support vector machine. Machine learning. Academic Press, London, pp 101–121

    Chapter  Google Scholar 

  • Quinto B (2020) Supervised learning, next-generation machine learning with spark: covers XGBoost, LightGBM, Spark NLP, distributed deep learning with keras, and more. Apress Berkeley, CA, pp 97–187

    Google Scholar 

  • Rahbarzare A, Azadi M (2019) Improving prediction of soil liquefaction using hybrid optimization algorithms and a fuzzy support vector machine. Bull Eng Geol Environ 78(7):4977–4987

    Article  Google Scholar 

  • Robertson PK, Wride C (1998) Evaluating cyclic liquefaction potential using the cone penetration test. Can Geotech J 35(3):442–459

    Article  Google Scholar 

  • Saeys Y, Inza I, Larranaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517

    Article  Google Scholar 

  • Sammut C, Webb GI (2011) Encyclopedia of machine learning. Springer Science & Business Media, NY

    Google Scholar 

  • Samui P (2011) Least square support vector machine and relevance vector machine for evaluating seismic liquefaction potential using SPT. Nat Hazards 59(2):811–822

    Article  Google Scholar 

  • Samui P, Sitharam T (2011) Machine learning modelling for predicting soil liquefaction susceptibility. Nat Hazard Earth Sys Sci 11(1):1–9

    Article  Google Scholar 

  • Sanz H, Valim C, Vegas E, Oller JM, Reverter F (2018) SVM-RFE: selection and visualization of the most relevant features through non-linear kernels. BMC Bioinformatics 19(1):1–18

    Article  Google Scholar 

  • Seed HB, Idriss IM (1971) Simplified procedure for evaluating soil liquefaction potential. J Soil Mech Found Div 97(9):1249–1273

    Article  Google Scholar 

  • Sinclair C, Pierce L, Matzner S (1999) An application of machine learning to network intrusion detection. In: Proceedings 15th Annual Computer Security Applications Conference (ACSAC'99). IEEE, pp. 371–377.

  • Sukkarak R, Tanapalungkorn W, Likitlersuang S, Ueda K (2021) Liquefaction analysis of sandy soil during strong earthquake in Northern Thailand. Soils Found 61(5):1302–1318

    Article  Google Scholar 

  • Suthaharan S (2016) Support vector machine, machine learning models and algorithms for big data classification. Springer, Boston, pp 207–235

    Google Scholar 

  • Syarif I, Prugel-Bennett A, Wills G (2016) SVM parameter optimization using grid search and genetic algorithm to improve classification performance. Telkomnika 14(4):1502

    Article  Google Scholar 

  • Thangavel K, Sasirekha K (2022). Classification of COVID-19 chest CT images using optimized deep convolutional generative adversarial network and deep CNN. In: Congress on Intelligent Systems: Proceedings of CIS 2021, vol. 2, Springer Nature.

  • Thompson SK (2012) Simple random sampling. In: Sampling, 3rd ed. Wiley Series in probability and statistics, pp. 9–37.

  • Tokimatsu K, Katsumata K (2011) Liquefaction-induced damage to buildings in Urayasu city during the 2011 Tohoku Pacific earthquake. In: Proceedings of the international symposium on engineering lessons learned from the, pp. 665–674.

  • Turlapati VPK, Prusty MR (2020) Outlier-SMOTE: a refined oversampling technique for improved detection of COVID-19. Intell-Based Med 3–4:100023

    Article  Google Scholar 

  • Van Ballegooy S, Malan P, Lacrosse V, Jacka M, Cubrinovski M, Bray J, O’Rourke T, Crawford S, Cowan H (2014) Assessment of liquefaction-induced land damage for residential Christchurch. Earthq Spect 30(1):31–55

    Article  Google Scholar 

  • Vapnik V (1999) The nature of statistical learning theory. Springer Science & Business Media, NY

    Google Scholar 

  • Vijayanand R, Devaraj D (2020) A novel feature selection method using whale optimization algorithm and genetic operators for intrusion detection system in wireless mesh network. IEEE Access 8:56847–56854

    Article  Google Scholar 

  • Vluymans S (2018) Dealing with imbalanced and weakly labelled data in machine learning using fuzzy and rough set methods, Ph.D. thesis. Ghent University, Faculty of Medicine and Health Sciences; University of Granada. Department of Computer Science and Artificial Intelligence, Ghent, Belgium; Granada, Spain, Springer.

  • XGBoost-Documentation (2021) XGBoost, URL: https://xgboost.readthedocs.io/en/stable/index.html#. Accessed on 18 Jan 2022.

  • Xue X, Liu E (2017) Seismic liquefaction potential assessed by neural networks. Environ Earth Sci 76(5):1–15

    Article  Google Scholar 

  • Xue X, Xiao M (2016) Application of genetic algorithm-based support vector machines for prediction of soil liquefaction. Environ Earth Sci 75(10):1–11

    Article  Google Scholar 

  • Yasuda S, Harada K, Ishikawa K, Kanemaru Y (2012) Characteristics of liquefaction in Tokyo Bay area by the 2011 Great East Japan earthquake. Soils Found 52(5):793–810

    Article  Google Scholar 

  • Youd TL, DeDen DW, Bray JD, Sancio R, Cetin KO, Gerber TM (2009) Zero-displacement lateral spreads, 1999 Kocaeli, Turkey, earthquake. J Geotech Geoenviron Eng 135(1):46–61

    Article  Google Scholar 

  • Zhang H, Qiu D, Wu R, Deng Y, Ji D, Li T (2019) Novel framework for image attribute annotation with gene selection XGBoost algorithm and relative attribute model. Applied Soft Comput 80:57–79

    Article  Google Scholar 

  • Zhang W, Wu C, Zhong H, Li Y, Wang L (2021a) Prediction of undrained shear strength using extreme gradient boosting and random forest based on Bayesian optimization. Geosci Front 12(1):469–477

    Article  Google Scholar 

  • Zhang X, Chen W, Wang B, Chen X (2015) Intelligent fault diagnosis of rotating machinery using support vector machine with ant colony algorithm for synchronous feature selection and parameter optimization. Neurocomputing 167:260–279

    Article  Google Scholar 

  • Zhang Y, Qiu J, Zhang Y, Wei Y (2021b) The adoption of ELM to the prediction of soil liquefaction based on CPT. Nat Hazards 107(1):539–549

    Article  Google Scholar 

  • Zhang Y, Qiu J, Zhang Y, Xie Y (2021c) The adoption of a support vector machine optimized by GWO to the prediction of soil liquefaction. Environ Earth Sci 80(9):1–9

    Article  Google Scholar 

  • Zhang Y, Xie Y, Zhang Y, Qiu J, Wu S (2021d) The adoption of deep neural network (DNN) to the prediction of soil liquefaction based on shear wave velocity. Bull Eng Geol Environ 80(6):5053–5060

    Article  Google Scholar 

  • Zhou J, Huang S, Wang M, Qiu Y (2021) Performance evaluation of hybrid GA–SVM and GWO–SVM models to predict earthquake-induced liquefaction potential of soil: a multi-dataset investigation. Eng Comput. https://doi.org/10.1007/s00366-021-01418-3

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Department of Civil Engineering, Bolu Abant Izzet Baysal University, 14030, Bolu, Turkey

    Selçuk Demir & Emrehan Kutluğ Şahin

Authors
  1. Selçuk Demir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emrehan Kutluğ Şahin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Selçuk Demir.

Ethics declarations

Conflicts of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Demir, S., Şahin, E.K. Liquefaction prediction with robust machine learning algorithms (SVM, RF, and XGBoost) supported by genetic algorithm-based feature selection and parameter optimization from the perspective of data processing. Environ Earth Sci 81, 459 (2022). https://doi.org/10.1007/s12665-022-10578-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12665-022-10578-4

Keywords

Search

Navigation