Abstract
目的
对钻进效率进行精确预测是制定土方开挖进度计划的关键。但现有预测方法多采用单个机器学习模型, 存在参数敏感性和过拟合等问题, 且往往忽略了环境因素和人员操作因素的影响。针对这些问题, 本文提出一种同时考虑多种因素综合影响的新的集成学习预测方法。
创新点
1. 建立一种基于Stacking集成学习的钻进效率预测模型; 2. 定量地考虑地质特性、人员操作、环境和机械特性等多种因素的综合影响; 3. 提出一种基于自适应步长策略的改进布谷鸟搜索优化方法 , 优化模型关键参数。
方法
1. 通过多次对比实验, 最终选择极值梯度提升(XGBoost)、随机森林(RF)和反向传播神经网络(BPNN)三个模型作为基学习器, 支持向量回归(SVR)作为元学习器进行集成。2. 建立基于自适应步长策略的改进布谷鸟搜索优化算法, 对集成模型的Max_depth等超参数进行优化。3. 将钻进效率值及相关影响因素的样本数据输入到每个基学习器中, 得到相应的输出结果, 再将预测结果作为元学习器的输入值, 得到最终的预测结果。4. 以中国西南地区某土石方工程为例, 通过五折交叉验证方法, 验证模型的鲁棒性, 并采用五个常用评价指标评价模型的精度和泛化性能。
结论
工程应用结果表明, 相比于目前流行的单个机器学习方法中预测性能最好的XGBoost和基于粒子群算法优化的Stacking集成模型, 本文所提方法的平均绝对百分比误差(MAPE)分别提高了16.43%和4.88%。
Abstract
Accurate prediction of drilling efficiency is critical for developing the earth-rock excavation schedule. The single machine learning (ML) prediction models usually suffer from problems including parameter sensitivity and overfitting. In addition, the influence of environmental and operational factors is often ignored. In response, a novel stacking-based ensemble learning method taking into account the combined effects of those factors is proposed. Through multiple comparison tests, four models, eXtreme gradient boosting (XGBoost), random forest (RF), back propagation neural network (BPNN) as the base learners, and support vector regression (SVR) as the meta-learner, are selected for stacking. Furthermore, an improved cuckoo search optimization (ICSO) algorithm is developed for hyper-parameter optimization of the ensemble model. The application to a real-world project demonstrates that the proposed method outperforms the popular single ML method XGBoost and the ensemble model optimized by particle swarm optimization (PSO), with 16.43% and 4.88% improvements of mean absolute percentage error (MAPE), respectively.
This is a preview of subscription content, log in via an institution to check access.
References
Abbas AK, Rushdi S, Alsaba M, et al., 2019. Drilling rate of penetration prediction of high-angled wells using artificial neural networks. Journal of Energy Resources Technology, 141(11):112904. https://doi.org/10.1115/1.4043699
Abbaspour H, Drebenstedt C, Badroddin M, et al., 2018. Optimized design of drilling and blasting operations in open pit mines under technical and economic uncertainties by system dynamic modelling. International Journal of Mining Science and Technology, 28(6):839–848. https://doi.org/10.1016/j.ijmst.2018.06.009
Abu Bakar MZ, Butt IA, Majeed Y, 2018. Penetration rate and specific energy prediction of rotary-percussive drills using drill cuttings and engineering properties of selected rock units. Journal of Mining Science, 54(2):270–284. https://doi.org/10.1134/S106273911802363X
Akün ME, Karpuz C, 2005. Drillability studies of surface-set diamond drilling in Zonguldak region sandstones from Turkey. International Journal of Rock Mechanics and Mining Sciences, 42(3):473–479. https://doi.org/10.1016/j.ijrmms.2004.11.009
Breiman L, 2001. Random forests. Machine Learning, 45(1): 5–32. https://doi.org/10.1023/A:1010933404324
Bui DT, Nhu VH, Hoang ND, 2018. Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and multi-layer perceptron neural network. Advanced Engineering Informatics, 38:593–604. https://doi.org/10.1016/j.aei.2018.09.005
Cankurt S, Subasi A, 2022. Tourism demand forecasting using stacking ensemble model with adaptive fuzzy combiner. Soft Computing, 26(7):3455–3467. https://doi.org/10.1007/s00500-021-06695-0
Chen KL, Jiang JC, Zheng FD, et al., 2018. A novel data-driven approach for residential electricity consumption prediction based on ensemble learning. Energy, 150:49–60. https://doi.org/10.1016/j.energy.2018.02.028
Chen WL, Wang XL, Cai ZJ, et al., 2021. DP-GMM clustering-based ensemble learning prediction methodology for dam deformation considering spatiotemporal differentiation. Knowledge-Based Systems, 222:106964. https://doi.org/10.1016/j.knosys.2021.106964
Cui SZ, Yin YQ, Wang DJ, et al., 2021. A stacking-based ensemble learning method for earthquake casualty prediction. Applied Soft Computing, 101:107038. https://doi.org/10.1016/j.asoc.2020.107038
Darbor M, Faramarzi L, Sharifzadeh M, 2019. Performance assessment of rotary drilling using non-linear multiple regression analysis and multilayer perceptron neural network. Bulletin of Engineering Geology and the Environment, 78(3):1501–1513. https://doi.org/10.1007/s10064-017-1192-3
Elkatatny S, 2018. Application of artificial intelligence techniques to estimate the static Poisson’s ratio based on wireline log data. Journal of Energy Resources Technology, 140(7):072905. https://doi.org/10.1115/1.4039613
Galar M, Fernandez A, Barrenechea E, et al., 2012. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):463–484. https://doi.org/10.1109/TSMCC.2011.2161285
Gan C, Cao WH, Wu M, et al., 2019. Prediction of drilling rate of penetration (ROP) using hybrid support vector regression: a case study on the Shennongjia area, central China. Journal of Petroleum Science and Engineering, 181:106200. https://doi.org/10.1016/j.petrol.2019.106200
Guo YY, Wang X, Xiao PC, et al., 2020. An ensemble learning framework for convolutional neural network based on multiple classifiers. Soft Computing, 24(5):3727–3735. https://doi.org/10.1007/s00500-019-04141-w
Haghighi F, Omranpour H, 2021. Stacking ensemble model of deep learning and its application to Persian/Arabic handwritten digits recognition. Knowledge-Based Systems, 220:106940. https://doi.org/10.1016/j.knosys.2021.106940
Hustrulid WA, Kuchta M, Martin R, 2013. Open Pit Mine Planning and Design, 3rd Edition. Taylor & Francis, Boca Raton, Florida, USA.
Kahraman S, 2002. Correlation of TBM and drilling machine performances with rock brittleness. Engineering Geology, 65(4):269–283. https://doi.org/10.1016/s0013-7952(01)00137-5
Kahraman S, Bilgin N, Feridunoglu C, 2003. Dominant rock properties affecting the penetration rate of percussive drills. International Journal of Rock Mechanics and Mining Sciences, 40(5):711–723. https://doi.org/10.1016/s1365-1609(03)00063-7
Kaushik A, Kaur P, Choudhary N, et al., 2022. Stacking regularization in analogy-based software effort estimation. Soft Computing, 26(3):1197–1216. https://doi.org/10.1007/s00500-021-06564-w
Kazemzadeh MR, Amjadian A, Amraee T, 2020. A hybrid data mining driven algorithm for long term electric peak load and energy demand forecasting. Energy, 204:117948. https://doi.org/10.1016/j.energy.2020.117948
Koopialipoor M, Tootoonchi H, Armaghani DJ, et al., 2019. Application of deep neural networks in predicting the penetration rate of tunnel boring machines. Bulletin of Engineering Geology and the Environment, 78(8):6347–6360. https://doi.org/10.1007/s10064-019-01538-7
Li H, Wang XS, Ding SF, 2018. Research and development of neural network ensembles: a survey. Artificial Intelligence Review, 49(4):455–479. https://doi.org/10.1007/s10462-016-9535-1
Li LL, Cen ZY, Tseng ML, et al., 2021. Improving short-term wind power prediction using hybrid improved cuckoo search arithmetic-support vector regression machine. Journal of Cleaner Production, 279:123739. https://doi.org/10.1016/j.jclepro.2020.123739
Li SP, 2018. Research on Excavation Simulation of a Pumped Storage Power Station Based on Excavation and Filling Balance. MS Thesis, Tianjin University, Tianjin, China (in Chinese).
Li ZX, Wu DZ, Hu C, et al., 2019. An ensemble learning-based prognostic approach with degradation-dependent weights for remaining useful life prediction. Reliability Engineering & System Safety, 184:110–122. https://doi.org/10.1016/j.ress.2017.12.016
Lv F, Wang JJ, Cui B, et al., 2020. An improved extreme gradient boosting approach to vehicle speed prediction for construction simulation of earthwork. Automation in Construction, 119:103351. https://doi.org/10.1016/j.autcon.2020.103351
Mao CY, Lin RR, Towey D, et al., 2021. Trustworthiness prediction of cloud services based on selective neural network ensemble learning. Expert Systems with Applications, 168:114390. https://doi.org/10.1016/j.eswa.2020.114390
Mendes-Moreira J, Soares C, Jorge AM, et al., 2012. Ensemble approaches for regression: a survey. ACM Computing Surveys, 45(1):10. https://doi.org/10.1145/2379776.2379786
Meng XJ, Chang JX, Wang XB, et al., 2019. Multi-objective hydropower station operation using an improved cuckoo search algorithm. Energy, 168:425–439. https://doi.org/10.1016/j.energy.2018.11.096
Mustafa AB, Abbas AK, Alsaba M, et al., 2021. Improving drilling performance through optimizing controllable drilling parameters. Journal of Petroleum Exploration and Production, 11(3):1223–1232. https://doi.org/10.1007/s13202-021-01116-2
Paul A, Bhowmik S, Panua R, et al., 2018. Artificial neural network-based prediction of performances-exhaust emissions of diesohol piloted dual fuel diesel engine under varying compressed natural gas flowrates. Journal of Energy Resources Technology, 140(11):112201. https://doi.org/10.1115/1.4040380
Pavlyukevich I, 2007. Lévy flights, non-local search and simulated annealing. Journal of Computational Physics, 226(2):1830–1844. https://doi.org/10.1016/j.jcp.2007.06.008
Peng H, Zeng ZG, Deng CS, et al., 2021. Multi-strategy serial cuckoo search algorithm for global optimization. Knowledge-Based Systems, 214:106729. https://doi.org/10.1016/j.knosys.2020.106729
Pernía-Espinoza A, Fernandez-Ceniceros J, Antonanzas J, et al., 2018. Stacking ensemble with parsimonious base models to improve generalization capability in the characterization of steel bolted components. Applied Soft Computing, 70:737–750. https://doi.org/10.1016/j.asoc.2018.06.005
Qi CC, Fourie A, Chen QS, 2018. Neural network and particle swarm optimization for predicting the unconfined compressive strength of cemented paste backfill. Construction and Building Materials, 159:473–478. https://doi.org/10.1016/j.conbuildmat.2017.11.006
Ren QB, Li MC, Song LG, et al., 2020. An optimized combination prediction model for concrete dam deformation considering quantitative evaluation and hysteresis correction. Advanced Engineering Informatics, 46:101154. https://doi.org/10.1016/j.aei.2020.101154
Saeidi O, Torabi SR, Ataei M, et al., 2014. A stochastic penetration rate model for rotary drilling in surface mines. International Journal of Rock Mechanics and Mining Sciences, 68:55–65. https://doi.org/10.1016/j.ijrmms.2014.02.007
Salimi A, Rostami J, Moormann C, et al., 2016. Application of non-linear regression analysis and artificial intelligence algorithms for performance prediction of hard rock TBMs. Tunnelling and Underground Space Technology, 58:236–246. https://doi.org/10.1016/j.tust.2016.05.009
Shi X, Liu G, Gong XL, et al., 2016. An efficient approach for real-time prediction of rate of penetration in offshore drilling. Mathematical Problems in Engineering, 2016:3575380. https://doi.org/10.1155/2016/3575380
Wang H, Zhang YM, Mao JX, et al., 2020. A probabilistic approach for short-term prediction of wind gust speed using ensemble learning. Journal of Wind Engineering and Industrial Aerodynamics, 202:104198. https://doi.org/10.1016/j.jweia.2020.104198
Wang L, Zeng Y, Chen T, 2015. Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications, 42(2):855–863. https://doi.org/10.1016/j.eswa.2014.08.018
Wang N, Zhao SY, Cui SZ, et al., 2021. A hybrid ensemble learning method for the identification of gang-related arson cases. Knowledge-Based Systems, 218:106875. https://doi.org/10.1016/j.knosys.2021.106875
Wang RC, Wu S, 2019. Neural network model based prediction of fragmentation of blasting using the Levenberg-Marquardt algorithm. Journal of Hydroelectric Engineering, 38(7): 100–109 (in Chinese). https://doi.org/10.11660/slfdxb.20190710
Wang T, Li ZJ, Yan YJ, et al., 2007. A survey of fuzzy decision tree classifier methodology. Proceedings of the Second International Conference of Fuzzy Information and Engineering, p.959–968. https://doi.org/10.1007/978-3-540-71441-5_104
Yan T, Shen SL, Zhou AN, et al., 2022. Prediction of geological characteristics from shield operational parameters by integrating grid search and K-fold cross validation into stacking classification algorithm. Journal of Rock Mechanics and Geotechnical Engineering, 14(4):1292–1303. https://doi.org/10.1016/j.jrmge.2022.03.002
Yang XS, Deb S, 2014. Cuckoo search: recent advances and applications. Neural Computing and Applications, 24(1): 169–174. https://doi.org/10.1007/s00521-013-1367-1
Zhang XH, Zhu QX, He YL, et al., 2018. A novel robust ensemble model integrated extreme learning machine with multi-activation functions for energy modeling and analysis: application to petrochemical industry. Energy, 162: 593–602. https://doi.org/10.1016/j.energy.2018.08.069
Acknowledgments
This work is supported by the Yalong River Joint Funds of the National Natural Science Foundation of China (No. U1965207) and the National Natural Science Foundation of China (Nos. 51839007, 51779169, and 52009090).
Author information
Authors and Affiliations
Corresponding author
Additional information
Author contributions
Jia YU designed the research. Fei LV processed the corresponding data. Fei LV and Jun ZHANG wrote the first draft of the manuscript. Peng YU and Da-wei TONG helped to organize the manuscript. Jia YU and Bin-ping WU revised and edited the final version.
Conflict of interest
Fei LV, Jia YU, Jun ZHANG, Peng YU, Da-wei TONG, and Bin-ping WU declare that they have no conflict of interest.
Electronic supplementary materials
Figs. S1–S4, Tables S1–S6, and Section S1
Electronic supplementary materials
Rights and permissions
About this article
Cite this article
Lv, F., Yu, J., Zhang, J. et al. A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation. J. Zhejiang Univ. Sci. A 23, 1027–1046 (2022). https://doi.org/10.1631/jzus.A2200297
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.A2200297
Key words
- Drilling efficiency
- Prediction
- Earth-rock excavation
- Stacking-based ensemble learning
- Improved cuckoo search optimization (ICSO) algorithm
- Comprehensive effects of various factors
- Hyper-parameter optimization