Skip to main content
SpringerLink
Log in

Explainable Boosting Machine for Predicting Wind Shear-Induced Aircraft Go-around based on Pilot Reports

  • Environmental Engineering
  • Published:
KSCE Journal of Civil Engineering Aims and scope

Abstract

The go-around is a safety-critical procedure in civil aviation that is rarely executed but is essential to avoid risky landings. Analyzing the factors that trigger go-around events can aid in identifying measures that could lower their frequency. This involves circumstances that could be deemed abnormal and intrinsically harmful. The study employed the Explainable Boosting Machine (EBM), a contemporary transparent model, to predict aircraft go-arounds and interpret different influential factors. The model proposed exhibits comparable accuracy to black-box models. The study utilized pilot reports and applied SMOTE-ENN to address the imbalance problem. The EBM model was trained with treated data in conjunction with Bayesian optimization. The EBM model’s performance was evaluated using holdout data and compared to binary logistic regression and decision tree models, as well as black-box models such as adaptive boosting, random forest, and extreme gradient boosting. The EBM model exhibited superior performance compared to other models in terms of precision (83.15%), recall (79.77%), geometric mean (77.29%), and Matthews’s correlation coefficient (0.453). The EBM algorithm enables the comprehensive interpretation of individual and pairwise factor interactions in predicting aircraft go-around outcomes from both global and local perspectives. This facilitates the assessment of the impact of different factors on go-around outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Blajev T, Curtis W (2017) Go-around decision-making and execution project. Final Report to Flight Safety Foundation

  • Bosschieter TM, Xu Z, Lan H, Lengerich BJ, Nori H, Sitcov K, Souter V, Caruana R (2022) Using interpretable machine learning to predict maternal and fetal outcomes. arXiv preprint arXiv:2207.05322, DOI: https://doi.org/10.48550/arXiv.2207.05322

  • Casner SM (2010) Why don’t pilots submit more pilot weather reports (PIREPs)? The International Journal of Aviation Psychology 20(4): 347–374, DOI: https://doi.org/10.1080/10508414.2010.487015

    Article  Google Scholar 

  • Causse M, Dehais F, Péran P, Sabatini U, Pastor J (2013) The effects of emotion on pilot decision-making: A neuroergonomic approach to aviation safety. Transportation Research Part C: Emerging Technologies 33:272–281, DOI: https://doi.org/10.1016/j.trc.2012.04.005

    Article  Google Scholar 

  • Chan PW (2012) A significant wind shear event leading to aircraft diversion at the Hong Kong international airport. Meteorological Applications 19(1):10–16, DOI: https://doi.org/10.1002/met.242

    Article  Google Scholar 

  • Chan PW (2022) Case study of a special event of low-level windshear and turbulence at the Hong Kong International Airport. Atmospheric Science Letters e1143, DOI: https://doi.org/10.1002/asl.1143

  • Chan PW, Hon KK (2016) Observation and numerical simulation of terrain-induced windshear at the Hong Kong International Airport in a planetary boundary layer without temperature inversions. Advances in Meteorology, DOI: https://doi.org/10.1155/2016/1454513

    Google Scholar 

  • Chan PW, Hon KK (2022) Observations and numerical simulations of sea breezes at Hong Kong International Airport. Weather, DOI: https://doi.org/10.1002/wea.4282

  • Chen F, Peng H, Chan PW, Huang Y, Hon KK (2022) Identification and analysis of terrain-induced low-level windshear at Hong Kong International Airport based on WRF–LES combining method. Meteorology and Atmospheric Physics 134(4):60, DOI: https://doi.org/10.1007/s00703-022-00899-1

    Article  Google Scholar 

  • Chen F, Peng H, Chan PW, Ma X, Zeng X (2020) Assessing the risk of windshear occurrence at HKIA using rare-event logistic regression. Meteorological Applications 27(6):e1962, DOI: https://doi.org/10.1002/met.1962

    Article  Google Scholar 

  • Chou CS, Tien A, Bateman H (2021) October. A machine learning application for predicting and alerting missed approaches for airport management. In 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) 1–9, IEEE, DOI: https://doi.org/10.1109/DASC52595.2021.9594418

  • Couronné R, Probst P, Boulesteix AL (2018) Random forest versus logistic regression: A large-scale benchmark experiment. BMC Bioinformatics 19:1–14, DOI: https://doi.org/10.1186/s12859-018-2264-5

    Article  Google Scholar 

  • Dai L, Liu Y, Hansen M (2021) Modeling go-around occurrence using principal component logistic regression. Transportation Research Part C: Emerging Technologies 129:103262, DOI: https://doi.org/10.1016/j.trc.2021.103262

    Article  Google Scholar 

  • Dao DV, Ly HB, Vu HLT, Le TT, Pham BT (2020) Investigation and optimization of the C-ANN structure in predicting the compressive strength of foamed concrete. Materials 13(5):1072, DOI: https://doi.org/10.3390/ma13051072

    Article  Google Scholar 

  • Dehais F, Behrend J, Peysakhovich V, Causse M, Wickens CD (2017) Pilot flying and pilot monitoring’s aircraft state awareness during go-around execution in aviation: A behavioral and eye tracking study. The International Journal of Aerospace Psychology 27(1–2):15–28, DOI: https://doi.org/10.1080/10508414.2017.1366269

    Article  Google Scholar 

  • Demir S, Sahin EK (2022) An investigation of feature selection methods for soil liquefaction prediction based on tree-based ensemble algorithms using AdaBoost, gradient boosting, and XGBoost. Neural Computing and Applications 1–18, DOI: https://doi.org/10.1007/s00521-022-07856-4

  • Donavalli B, Mattingly SP, Massidda A (2017) Impact of Weather Factors on Go-Around Frequency (No. 17-03934)

  • Dong S, Khattak A, Ullah I, Zhou J, Hussain A (2022) Predicting and analyzing road traffic injury severity using boosting-based ensemble learning models with SHAPley Additive exPlanations. International Journal of Environmental Research and Public Health 19(5):2925, DOI: https://doi.org/10.3390/ijerph19052925

    Article  Google Scholar 

  • Duan P, Soleimani H, Ghazanfari A, Dehghani M (2020) Distributed energy management in smart grids based on cloud-fog layer architecture considering PHEVs. IEEE Transactions on Industry Applications, DOI: https://doi.org/10.1109/TIA.2020.3010899

  • Ekanayake IU, Meddage DPP, Rathnayake U (2022) A novel approach to explain the black-box nature of machine learning in compressive strength predictions of concrete using Shapley additive explanations (SHAP). Case Studies in Construction Materials 16:e01059, DOI: https://doi.org/10.1016/j.cscm.2022.e01059

    Article  Google Scholar 

  • ElShawi R, Sherif Y, Al-Mallah M, Sakr S (2021) Interpretability in healthcare: A comparative study of local machine learning interpretability techniques. Computational Intelligence 37(4):1633–1650, DOI: https://doi.org/10.1109/CBMS.2019.00065

    Article  MathSciNet  Google Scholar 

  • Fitriyani NL, Syafrudin M, Alfian G, Yang CK, Rhee J, Ulyah SM (2022, June) Chronic disease prediction model using integration of DBSCAN, SMOTE-ENN, and random forest. In 2022 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) 289–294, IEEE, DOI: https://doi.org/10.1109/ICETSIS55481.2022.9888806

  • Frazier PI (2018) A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811, DOI: https://doi.org/10.48550/arXiv.1807.02811

  • Greener JG, Kandathil SM, Moffat L, Jones DT (2022) A guide to machine learning for biologists. Nature Reviews Molecular Cell Biology 23(1):40–55, DOI: https://doi.org/10.1038/s41580-021-00407-0

    Article  Google Scholar 

  • Han L, Yang G, Yang X, Song X, Xu B, Li Z, Wu J (2022) An explainable XGBoost model improved by SMOTE-ENN technique for maize lodging detection based on multi-source unmanned aerial vehicle images. Computers and Electronics in Agriculture 194:106804, DOI: https://doi.org/10.1016/j.compag.2022.106804

    Article  Google Scholar 

  • Hon KK (2020) Predicting low-level wind shear using 200-m-resolution NWP at the Hong Kong International Airport. Journal of Applied Meteorology and Climatology 59(2):193–206, DOI: https://doi.org/10.1175/JAMC-D-19-0186.1

    Article  MathSciNet  Google Scholar 

  • ICAO (2017) World airport traffic forecast 2017–2040 Airport Council International. Airport Council International: Montréal, QC, USA

    Google Scholar 

  • Isayev O, Oses C, Toher C, Gossett E, Curtarolo S, Tropsha A (2017) Universal fragment descriptors for predicting properties of inorganic crystals. Nature Communications 8(1):15679, DOI: https://doi.org/10.1038/ncomms15679

    Article  Google Scholar 

  • Jayasundara S, Indika A, Herath D (2022) Interpretable student performance prediction using explainable boosting machine for multi-class classification. 2022 2nd International Conference on Advanced Research in Computing (ICARC) 391–396. IEEE, DOI: https://doi.org/10.1109/ICARC54489.2022.9753867

  • Jou RC, Kuo CW, Tang ML (2013) A study of job stress and turnover tendency among air traffic controllers: The mediating effects of job satisfaction. Transportation Research Part E: Logistics and Transportation Review 57:95–104, DOI: https://doi.org/10.1016/j.tre.2013.01.009

    Article  Google Scholar 

  • Kennedy Q, Taylor JL, Reade G, Yesavage JA (2010) Age and expertise effects in aviation decision making and flight control in a flight simulator. Aviation, Space, and Environmental Medicine 81(5):489–497, DOI: https://doi.org/10.3357/asem.2684.2010

    Article  Google Scholar 

  • Khattak A, Almujibah H, Elamary A, Matara CM (2022) Interpretable dynamic ensemble selection approach for the prediction of road traffic injury severity: A case study of pakistan’s national highway N-5. Sustainability 14(19):12340, DOI: https://doi.org/10.3390/su141912340

    Article  Google Scholar 

  • Khattak A, Chan PW, Chen F, Peng H (2022) Prediction of aircraft go-around during wind shear using the dynamic ensemble selection framework and pilot reports. Atmosphere 13(12):2104, DOI: https://doi.org/10.3390/atmos13122104

    Article  Google Scholar 

  • Khattak A, Chan PW, Chen F, Peng H (2023) Prediction of a pilot’s invisible foe: The severe low-level wind shear. Atmosphere 14(1): 37, DOI: https://doi.org/10.3390/atmos14010037

    Article  Google Scholar 

  • Kim Y, Kim Y (2022) Explainable heat-related mortality with random forest and SHapley Additive exPlanations (SHAP) models. Sustainable Cities and Society 79:103677, DOI: https://doi.org/10.1016/j.scs.2022.103677

    Article  Google Scholar 

  • Kumarakulasinghe NB, Blomberg T, Liu J, Leao AS, Papapetrou P (2020, July) Evaluating local interpretable model-agnostic explanations on clinical machine learning classification models. In 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) 7–12, IEEE, DOI: https://doi.org/10.1109/CBMS49503.2020.00009

    Google Scholar 

  • Lee O, Shun CM (2003) Observation of sea breeze interactions at and near Hong Kong International Airport. Meteorological Applications 10(1):1–9, DOI: https://doi.org/10.1017/S1350482703005012

    Article  Google Scholar 

  • Lin M, Zhu X, Hua T, Tang X, Tu G, Chen X (2021) Detection of ionospheric scintillation based on xgboost model improved by SMOTE-ENN technique. Remote Sensing 13(13):2577, DOI: https://doi.org/10.3390/rs13132577

    Article  Google Scholar 

  • Liu G, Sun B (2023) Concrete compressive strength prediction using an explainable boosting machine model. Case Studies in Construction Materials e01845, DOI: https://doi.org/10.1016/j.cscm.2023.e01845

  • Liu H, Zhao Y, Zaporowska A, Skaf Z (2020) A machine learning-based clustering approach to diagnose multi-component degradation of aircraft fuel systems. Neural Computing and Applications 1–17, DOI: https://doi.org/10.1007/s00521-021-06531-4

  • Lou Y, Caruana R, Gehrke J, Hooker G (2013) Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 623–631, DOI: https://doi.org/10.1145/2487575.2487579

  • Luengo J, Fernández A, García S, Herrera F (2011) Addressing data complexity for imbalanced data sets: Analysis of SMOTE-based oversampling and evolutionary undersampling. Soft Computing 15:1909–1936, DOI: https://doi.org/10.1007/s00500-010-0625-8

    Article  Google Scholar 

  • Maddock S, Cormode G, Wang T, Maple C, Jha S (2022) Federated boosted decision trees with differential privacy. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security 2249–2263, DOI: https://doi.org/10.1145/3548606.3560687

  • Mangalathu S, Hwang SH, Jeon JS (2020) Failure mode and effects analysis of RC members based on machine-learning-based SHapley Additive exPlanations (SHAP) approach. Engineering Structures 219:110927, DOI: https://doi.org/10.1016/j.engstruct.2020.110927

    Article  Google Scholar 

  • McCarthy J, Serafin R, Wilson J, Evans J, Kessinger C, Mahoney III WP (2022) Addressing the microburst threat to aviation: research-to-operations success story. Bulletin of the American Meteorological Society 103(12):E2845–E2861, DOI: https://doi.org/10.1175/BAMS-D-22-0038.1

    Article  Google Scholar 

  • Mohril RS, Solanki BS, Kulkarni MS, Lad BK (2022) XGBoost based residual life prediction in the presence of human error in maintenance. Neural Computing and Applications 1–15, DOI: https://doi.org/10.1007/s00521-022-07216-2

  • Muntasir Nishat M, Faisal F, Jahan Ratul I, Al-Monsur A, Ar-Rafi AM, Nasrullah SM, Reza Md T, Khan Md RH (2022) A comprehensive investigation of the performances of different machine learning classifiers with SMOTE-ENN oversampling technique and hyperparameter optimization for imbalanced heart failure dataset. Scientific Programming 2022, 1–17, DOI: https://doi.org/10.1155/2022/3649406

  • Nohara Y, Matsumoto K, Soejima H, Nakashima N (2022) Explanation of machine learning models using shapley additive explanation and application for real data in hospital. Computer Methods and Programs in Biomedicine 214:106584, DOI: https://doi.org/10.1016/j.cmpb.2021.106584

    Article  Google Scholar 

  • Nori H, Jenkins S, Koch P, Caruana R (2019) Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223, DOI: https://doi.org/10.48550/arXiv.1909.09223

  • Prats X, Puig V, Quevedo J, Nejjari F (2010) Multi-objective optimisation for aircraft departure trajectories minimising noise annoyance. Transportation Research Part C: Emerging Technologies 18(6):975–989, DOI: https://doi.org/10.1016/j.trc.2010.03.001

    Article  Google Scholar 

  • Qu Y, Deng X, Lin S, Han F, Chang HH, Ou Y, Nie Z, Mai J, Wang X, Gao X, Wu Y, Chen J, Zhuang J, Ryan I, Liu X (2022) Using innovative machine learning methods to screen and identify predictors of congenital heart diseases. Frontiers in Cardiovascular Medicine 8:2087, DOI: https://doi.org/10.3389/fcvm.2021.797002

    Article  Google Scholar 

  • Rai A (2020) Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science 48:137–141, DOI: https://doi.org/10.1007/s11747-019-00710-5

    Article  Google Scholar 

  • Santos JI, Martín Ó, Ahedo V, de Tiedra P, Galán JM (2022) Glass-box modeling for quality assessment of resistance spot welding joints in industrial applications. The International Journal of Advanced Manufacturing Technology, 1–16, DOI: https://doi.org/10.1007/s00170-022-10444-4

  • Shortle J, Sherry L (2013) A model for investigating the interaction between go-arounds and runway throughput. In 2013 Aviation Technology, Integration, and Operations Conference 4235, DOI: https://doi.org/10.2514/6.2013-4235

  • Singh NP, Goh SK, Alam S (2020) Real-time unstable approach detection using sparse variational gaussian process. In 2020 International Conference on Artificial Intelligence and Data Analytics for Air Transportation (AIDA-AT) 1–10, IEEE, DOI: https://doi.org/10.1109/AIDA-AT48540.2020.9049174

  • Tse SM, Chan PW, Wong WK (2014) A case study of missed approach of aircraft due to tailwind associated with thunderstorms. Meteorological Applications 21(1):50–61, DOI: https://doi.org/10.1002/met.1296

    Article  Google Scholar 

  • Victoria AH, Maragatham G (2021) Automatic tuning of hyperparameters using Bayesian optimization. Evolving Systems 12:217–223, DOI: https://doi.org/10.1007/s12530-020-09345-2

    Article  Google Scholar 

  • Wang X, Yang YQ, Liu SH, Hong XY, Sun XF, Shi JH (2020) Comparing different venous thromboembolism risk assessment machine learning models in Chinese patients. Journal of Evaluation in Clinical Practice 26(1):26–34, DOI: https://doi.org/10.1111/jep.13324

    Article  Google Scholar 

  • Xiaolin L, Qingyuan W, Panicker RC, Cardiff B, John D (2022) Binary ECG classification using explainable boosting machines for IoT edge devices. In 2022 29th IEEE International Conference on Electronics, Circuits and Systems (ICECS) 1–4, IEEE, DOI: https://doi.org/10.1109/ICECS202256217.2022.9970834

  • Zaal P, Campbell A, Schroeder JA, Shah S (2019) Validation of proposed go-around criteria under various environmental conditions. In AIAA Aviation 2019 Forum 2993, DOI: https://doi.org/10.2514/6.2019-2993

  • Zhang S, Khattak A, Matara CM, Hussain A, Farooq A (2022) Hybrid feature selection-based machine learning Classification system for the prediction of injury severity in single and multiple-vehicle accidents. PLoS One 17(2):e0262941, DOI: https://doi.org/10.1371/journal.pone.0262941

    Article  Google Scholar 

  • Zorkeflee M, Mohamed Din A, Ku-Mahamud KR (2015) Fuzzy and smote resampling technique for imbalanced data sets, 638–643

Download references

Acknowledgments

This research was funded by the Research Fund for International Young Scientists of the National Natural Science Foundation of China (Grant No. 52250410351), the National Foreign Expert Project (Grant No. QN2022133001L), and the National Natural Science Foundation of China (U1733113). We acknowledge the Hong Kong Observatory of Hong Kong International Airport for providing pilot reports for this study.

Author information

Authors and Affiliations

  1. Key Laboratory of Infrastructure Durability and Operation Safety in Airfield of CAAC, College of Transportation Engineering, Tongji University, Shanghai, 201804, China

    Afaq Khattak & Feng Chen

  2. Hong Kong Observatory, Kowloon, Hong Kong, China

    Pak-wai Chan

  3. Shanghai Research Center for Smart Mobility and Road Safety, Shanghai, 200092, China

    Haorong Peng

Authors
  1. Afaq Khattak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pak-wai Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haorong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khattak, A., Chan, Pw., Chen, F. et al. Explainable Boosting Machine for Predicting Wind Shear-Induced Aircraft Go-around based on Pilot Reports. KSCE J Civ Eng 27, 4115–4129 (2023). https://doi.org/10.1007/s12205-023-0410-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12205-023-0410-8

Keywords

Search

Navigation