Abstract
Machine learning (ML) holds significant potential for predicting soil properties in geotechnical design but at the same time poses challenges, including those of how to easily examine the performance of an algorithm and how to select an optimal algorithm. This study first comprehensively reviewed the application of ML algorithms in modelling soil properties for geotechnical design. The algorithms were categorized into several groups based on their principles, and the main characteristics of these ML algorithms were summarized. After that six representative algorithms are further detailed and selected for the creation of a ML-based tool with which to easily build ML-based models. Interestingly, automatic determination of the optimal configurations of ML algorithms is developed, with an evaluation of model accuracy, application of the developed ML model to the new data and investigation of relationships between the input variables and soil properties. Furthermore, a novel ranking index is proposed for the model comparison and selection, which evaluates a ML-based model from five aspects. Soil maximum dry density is selected as an example to allow examination of the performance of different ML algorithms, the applicability of the tool and the model ranking index to determining an optimal model.
Similar content being viewed by others
Abbreviations
- b i :
-
Bias vector of the ith hidden layer
- c :
-
Constant coefficient vector
- C :
-
Regularization parameter
- E :
-
Exponent matrix
- F :
-
Function set
- gen :
-
Number of iterations
- H i :
-
Output of the ith hidden layer
- m :
-
Number of datasets
- mtry :
-
Number of features at each node
- n :
-
Dimension of input variables
- n t :
-
Dimension of transformed variables
- ntree :
-
Number of decision trees
- N :
-
Stochastic calculation times
- p :
-
Dropout probability
- p c :
-
Probability of crossover
- p m :
-
Probability of mutation
- pop :
-
Size of population
- r :
-
Bernoulli distribution with probability of p
- W i :
-
Weight matrix of the ith hidden layer
- x i, max :
-
Maximum value of the variable xi
- x i , min :
-
Minimum value of the variable xi
- x norm :
-
Normalized value of a dataset
- X = (x 1, x 2, …, x n):
-
Matrix of input variables
- XT :
-
Matrix of transformed variables
- y a i :
-
Actual value of the output variable
- y p i :
-
Predicted value of the output variable
- \(\bar{y}_{i}^{a}\) :
-
Mean value of the actual output variable
- y = (y 1, y 2, …, y n):
-
Output of the output layer
- γ :
-
Kernel coefficient
- ξ :
-
Slack parameter (default value: 0.1).
- σ :
-
Activation function
- \({\mathbb{E}}\) :
-
Mean value of output
References
Yin JH (1999) Properties and behaviour of Hong Kong marine deposits with different clay contents. Can Geotech J 36(6):1085–1095
Nagaraj TS, Murthy BRS (1986) A critical reappraisal of compression index equations. Géotechnique 36(1):27–32
Ouyang Z, Mayne PW (2019) Modified NTH method for assessing effective friction angle of normally consolidated and overconsolidated clays from piezocone tests. J Geotech Geoenviron Eng 145(10):04019067
Hattab M, Hammad T, Fleureau JM (2015) Internal friction angle variation in a kaolin/montmorillonite clay mix and microstructural identification. Géotechnique 65(1):1–11
Yoon GL, Kim BT, Jeon SS (2004) Empirical correlations of compression index for marine clay from regression analysis. Can Geotech J 41(6):1213–1221
Kootahi K (2017) Simple index tests for assessing the recompression index of fine-grained soils. J Geotech Geoenviron Eng 143(4):06016027
Hayden CP, Purchase-Sanborn K, Dewoolkar M (2018) Comparison of site-specific and empirical correlations for drained residual shear strength. Géotechnique 68(12):1099–1108
Zhang P, Jin Y. F, Yin Z. Y, Yang Y (2020) Random forest based artificial intelligent model for predicting failure envelopes of caisson foundations in sand. Appl Ocean Res 101:102223
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Krizhevsky A, Sutskever I, Hinton G, ImageNet classification with deep convolutional neural networks, NIPS, 2012
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444
Shen S-L, Atangana Njock PG, Zhou A, Lyu H-M (2021) Dynamic prediction of jet grouted column diameter in soft soil using Bi-LSTM deep learning. Acta Geotech 16:303–315
Atangana Njock P.G., Shen S.-L., Zhou A., Lyu H.-M. (2020) Evaluation of soil liquefaction using AI technology incorporating a coupled ENN / t-SNE model. Soil Dyn Earthq Eng, 130:105988
Lin S.-S., Shen S.-L., Zhang N., Zhou A. (2021) Comprehensive environmental impact evaluation for concrete mixing station (CMS) based on improved TOPSIS method. Sustainable Cities and Society, 69:102838
Kohestani VR, Hassanlourad M (2016) Modeling the mechanical behavior of carbonate sands using artificial neural networks and support vector machines. Int J Geomech 16(1):04015038
Penumadu D, Zhao RD (1999) Triaxial compression behavior of sand and gravel using artificial neural networks (ANN). Comput Geotech 24:207–230
Zhang N., Shen S.-L., Zhou A., Jin Y.-F. (2021) Application of LSTM approach for modelling stress–strain behaviour of soil. Appl Soft Comput, 100:106959
Zhang P, Yin ZY (2021) A novel deep learning-based modelling strategy from image of particles to mechanical properties for granular materials with CNN and BiLSTM. Comput Meth Appl Mech Eng 382:113858
Zhang P, Wu HN, Chen R.P, Chan HT (2020) Hybrid meta-heuristic and machine learning algorithms for tunneling-induced settlement prediction: a comparative study. Tunnell Undergr Space Technol, 99:103383
Zhang P, Yin ZY, Jin YF, Chan T, Gao FP (2021) Intelligent modelling of clay compressibility using hybrid meta-heuristic and machine learning algorithms. Geosci Front 12(1):441–452
Qi CC, Tang XL (2018) Slope stability prediction using integrated metaheuristic and machine learning approaches: a comparative study. Comput Ind Eng 118:112–122
Feng Y, Cui N, Hao W, Gao L, Gong D (2019) Estimation of soil temperature from meteorological data using different machine learning models. Geoderma 338:67–77
Zhang P, Yin ZY, Jin YF (2021) State-of-the-art review of machine learning applications in constitutive modeling of soils. Arch Comput Method Eng. https://doi.org/10.1007/s11831-020-09524-z
Elbaz K, Shen SL, Zhou AN, Yin ZY, Lyu HM (2021) Prediction of disc cutter life during shield tunnelling with AI via incorporation of genetic algorithm into GMDH-type neural network. Engineering 7(2):238–251
Gal Y., Ghahramani Z. (2015) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. arXiv:1506.02142
Blundell C., Cornebise J., Kavukcuoglu K., Wierstra D. (2015) Weight uncertainty in neural networks. arXiv:1505.05424v2
Graves A. (2011) Practical variational inference for neural networks. NIPS
Zhang P, Jin YF, Yin ZY (2021) Machine learning–based uncertainty modelling of mechanical properties of soft clays relating to time-dependent behavior and its application. Int J Numer Anal Methods Geomech. https://doi.org/10.1002/nag.3215
Tan F, Zhou W-H, Yuen K-V (2018) Effect of loading duration on uncertainty in creep analysis of clay. Int J Numer Anal Methods Geomech 42(11):1235–1254
Zhou WH, Tan F, Yuen KV (2018) Model updating and uncertainty analysis for creep behavior of soft soil. Comput Geotech 100:135–143
Tan F, Zhou W-H, Yuen K-V (2016) Modeling the soil water retention properties of same-textured soils with different initial void ratios. J Hydrol 542:731–743
Zhou W-H, Yuen K-V, Tan F (2014) Estimation of soil–water characteristic curve and relative permeability for granular soils with different initial dry densities. Eng Geol 179:1–9
Zhang W, Goh ATC (2016) Multivariate adaptive regression splines and neural network models for prediction of pile drivability. Geosci Front 7(1):45–52
Cheng Z.-L, Zhou W.-H, Garg A (2020) Genetic programming model for estimating soil suction in shallow soil layers in the vicinity of a tree. Eng Geol, 268:105506
Yin ZY, Jin YF, Huang HW, Shen SL (2016) Evolutionary polynomial regression based modelling of clay compressibility using an enhanced hybrid real-coded genetic algorithm. Eng Geol 210:158–167
Wang R, Zhang K, Wang W, Meng Y, Yang L, Huang H (2020) Hydrodynamic landslide displacement prediction using combined extreme learning machine and random search support vector regression model. Eur J Environ Civ Eng:1–13
Samui P, Sitharam TG (2008) Least-square support vector machine applied to settlement of shallow foundations on cohesionless soils. Int J Numer Anal Met 32(17):419–427
Ai L, Fang NF, Zhang B, Shi ZH (2013) Broad area mapping of monthly soil erosion risk using fuzzy decision tree approach: integration of multi-source data within GIS. Int J Geogr Inf Sci 27(6):1251–1267
Qi C, Fourie A, Zhao X (2018) Back-analysis method for stope displacements using gradient-boosted regression tree and firefly algorithm. J Comput Civil Eng 32(5):04018031
Zhang P, Yin Z.Y, Jin YF, Chan THT (2020) A novel hybrid surrogate intelligent model for creep index prediction based on particle swarm optimization and random forest. Eng Geol, 265:105328
Sanikhani H, Deo RC, Yaseen ZM, Eray O, Kisi O (2018) Non-tuned data intelligent model for soil temperature estimation: a new approach. Geoderma 330:52–64
Yamaç SS, Şeker C, Negiş H (2020) Evaluation of machine learning methods to predict soil moisture constants with different combinations of soil input data for calcareous soils in a semi arid area. Agric Water Manage, 234:106121
Chen RP, Zhang P, Kang X, Zhong ZQ, Liu Y, Wu HN (2019) Prediction of maximum surface settlement caused by EPB shield tunneling with ANN methods. Soils Found 59(2):284–295
Yilmaz I, Marschalko M, Bednarik M, Kaynar O, Fojtova L (2012) Neural computing models for prediction of permeability coefficient of coarse-grained soils. Neural Comput Appl 21(5):957–968
Kiefa MAA (1998) General regression neural networks for driven piles in cohesionless soils. J Geotech Geoenviron Eng 124(12):1177–1185
Feng X, Jimenez R (2015) Predicting tunnel squeezing with incomplete data using Bayesian networks. Eng Geol 195:214–224
Goh ATC, Kulhawy FH, Chua CG (2005) Bayesian neural network analysis of undrained side resistance of drilled shafts. J Geotech Geoenviron Eng 131(1):84–93
Koza JR (1992) Genetic programming: on the programming of computers by natural selection, MIT Press. MA, Cambridge
Sette S, Boullart L (2001) Genetic programming: principles and applications. Eng Appl Artif Intel 14:727–736
Giustolisi O, Savic DA (2006) A symbolic data-driven technique based on evolutionary polynomial regression. J Hydroinform 8(4):235–237
Jin YF, Yin ZY (2020) Enhancement of backtracking search algorithm for identifying soil parameters. Int J Numer Anal Methods Geomech 44(9):1239–1261
Jin YF, Yin ZY (2020) An intelligent multi-objective EPR technique with multi-step model selection for correlations of soil properties. Acta Geotech 15(8):2053–2073
Yin ZY, Jin YF, Shen JS, Hicher PY (2018) Optimization techniques for identifying soil parameters in geotechnical engineering: comparative study and enhancement. Int J Numer Anal Methods Geomech 42(1):70–94
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297
Breiman L (2001) Random forests. Mach Learn 45:5–32
Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140
Zhang P, Chen RP, Wu HN (2019) Real-time analysis and regulation of EPB shield steering using Random Forest. Automat Constr, 106:102860
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(9):533–536
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958
Wang HL, Yin ZY (2020) High performance prediction of soil compaction parameters using multi expression programming. Eng Geol, 276:105758
Ören AH (2014) Estimating compaction parameters of clayey soils from sediment volume test. Appl Clay Sci 101:68–72
Gurtug Y, Sridharan A (2004) Compaction behaviour and prediction of its characteristics of fine grained soils with particular reference to compaction energy. Soils Found 44(5):27–36
AI-Khafaji A.N. (1993) Estimation of soil compaction parameters by means of Atterberg limits. Q J Eng Geol 26:359–368
Luo G (2016) A review of automatic selection methods for machine learning algorithms and hyper-parameter values. Netw Model Anal Health Inform Bioinforma 5:18
Zhang P, Yin ZY, Jin YF, Ye GL (2020) An AI-based model for describing cyclic characteristics of granular materials. Int J Numer Anal Methods Geomech 44(9):1315–1335
Zorlu K, Gokceoglu C, Ocakoglu F, Nefeslioglu HA, Acikalin S (2008) Prediction of uniaxial compressive strength of sandstones using petrography-based models. Eng Geol 96(3–4):141–158
Acknowledgements
This research was financially supported by the Research Grants Council (RGC) of Hong Kong Special Administrative Region Government (HKSARG) of China (Grant No.: 15220221, R5037-18F).
Author information
Authors and Affiliations
Contributions
PZ: conceptualization, methodology, analysis, writing-review and original draft. Z-Yu Y: supervision, methodology, visualization, writing-review and editing. Y-F J: validation, visualization and editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that the work described has not been published before; that it is not under consideration for publication anywhere else; that its publication has been approved by all co-authors; that there is no conflict of interest regarding the publication of this article.
Data Availability Statement
(GUI download available: https://www.researchgate.net/publication/348617390_ErosMLM). All data used during the study are available from the corresponding author by request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, P., Yin, ZY. & Jin, YF. Machine Learning-Based Modelling of Soil Properties for Geotechnical Design: Review, Tool Development and Comparison. Arch Computat Methods Eng 29, 1229–1245 (2022). https://doi.org/10.1007/s11831-021-09615-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11831-021-09615-5