Skip to main content
SpringerLink
Log in

Analyzing flight delay prediction under concept drift

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

Flight delays impose challenges that impact any flight transportation system-predicting when they will occur in a meaningful way to mitigate this issue. However, the distribution of the flight delay system variables changes over time. This phenomenon is known in predictive analytics as concept drift. This paper investigates the prediction performance of different drift handling strategies in aviation under different scales (models trained from flights related to a single airport or the entire flight system). Specifically, two research questions were proposed and answered: (1) how do drift handling strategies influence the prediction performance of delays? (2) Do different scales change the results of drift handling strategies? In our analysis, drift handling strategies are relevant, and their impacts vary according to scale and machine learning models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Search string used: (“flight delay”) and (“classification” or “regression” or “prediction”).

References

  • Ai Y, Pan W, Yang C, Wu D, Tang J (2019) A deep learning approach to predict the spatial and temporal distribution of flight delay in network. J Intell Fuzzy Syst 37(5):6029–6037

    Article  Google Scholar 

  • Alla H, Moumoun L, Balouki Y (2021) Flight arrival delay prediction using supervised machine learning algorithms. In: Gherabi N, Kacprzyk J (eds) Intelligent systems in big data, semantic web and machine learning, advances in intelligent systems and computing. Springer International Publishing, Cham, pp 231–246

    Google Scholar 

  • Alonso H, Loureiro A (2015) Predicting flight departure delay at porto airport: a preliminary study. In: 2015 7th International joint conference on computational intelligence (IJCCI). Lisbon, Portugal, pp 93–98

  • ANAC (2017) The Brazilian National Civil Aviation Agency. Tech. rep., http://www.anac.gov.br/

  • Angelov P, Zhou X (2008) On line learning fuzzy rule-based system structure from data streams. In: 2008 IEEE international conference on fuzzy systems (IEEE World Congress on computational intelligence). Hong Kong, China, pp 915–922

  • ASOS: Automated Surface Observing Systems (2019) Tech. rep., https://mesonet.agron.iastate.edu/request/download.phtml

  • Belcastro L, Marozzo F, Talia D, Trunfio P (2016) Using scalable data mining for predicting flight delays. ACM Trans Intell Syst Technol 8(1):1–20

    Article  Google Scholar 

  • Carvalho L, Sternberg A, Maia Gonçalves L, Beatriz Cruz A, Soares J, Brandão D, Carvalho D (2021) On the relevance of data science for flight delay research: a systematic review. Transp Rev 41(4):499–528

    Article  Google Scholar 

  • Chen H, Wang J, Yan X (2008) A fuzzy support vector machine with weighted margin for flight delay early warning. In: 2008 Fifth international conference on fuzzy systems and knowledge discovery. Shandong, China, pp 331–335

  • Du WB, Zhang MY, Zhang Y, Cao XB, Zhang J (2018) Delay causality network in air transport systems. Transp Res Part E Logist Transp Rev 118:466–476

    Article  Google Scholar 

  • Gama J, Zliobaite I, Bifet A, Pechenizkiy M, Bouchachia A (2014) A survey on concept drift adaptation. ACM Comput Surv 46(4):1–37

    Article  Google Scholar 

  • Groß J (2012) Linear regression. Springer Science & Business Media, Berlin

    Google Scholar 

  • Gui G, Liu F, Sun J, Yang J, Zhou Z, Zhao D (2020) Flight delay prediction based on aviation big data and machine learning. IEEE Trans Veh Technol 69(1):140–150

    Article  Google Scholar 

  • Guleria Y, Cai Q, Alam S, Li L (2019) A multi-agent approach for reactionary delay prediction of flights. IEEE Access 7:181565–181579

    Article  Google Scholar 

  • Han J, Pei J, Kamber M (2011) Data mining: concepts and techniques. Elsevier, New York

    MATH  Google Scholar 

  • Hoens T, Polikar R, Chawla N (2012) Learning from streaming data with concept drift and imbalance: an overview. Progr Artif Intell 1(1):89–101

    Article  Google Scholar 

  • Iwashita A, Papa J (2019) An overview on concept drift learning. IEEE Access 7:1532–1547

    Article  Google Scholar 

  • James G, Witten D, Hastie T, Tibshirani R (2013) An introduction to statistical learning: with applications in R. Springer Science & Business Media, Berlin

    Book  Google Scholar 

  • Khamassi I, Sayed-Mouchaweh M (2014) Drift detection and monitoring in non-stationary environments. In: 2014 IEEE conference on evolving and adaptive intelligent systems (EAIS). Linz, Austria, pp 1–6

  • Khanmohammadi S, Chou CA, Lewis HW, I, Elias D (2014) A systems approach for scheduling aircraft landings in JFK airport. In: 2014 IEEE international conference on fuzzy systems (FUZZ-IEEE). Beijing, China, pp 1578–1585

  • Kim Y, Choi S, Briceno S, Mavris D (2016) A deep learning approach to flight delay prediction. In: 2016 IEEE/AIAA 35th Digital avionics systems conference (DASC). Sacramento, CA, USA, pp 1–6

  • Lu J, Liu A, Dong F, Gu F, Gama J, Zhang G (2019) Learning under concept drift: a review. IEEE Trans Knowl Data Eng 31(12):2346–2363

    Article  Google Scholar 

  • Lughofer E, Angelov P (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput J 11(2):2057–2068

    Article  Google Scholar 

  • Moreira L, Dantas C, Oliveira L, Soares J, Ogasawara E (2018) On evaluating data preprocessing methods for machine learning models for flight delays. In: 2018 International joint conference on neural networks (IJCNN). Rio de Janeiro, Brazil, pp 1–8

  • Munoz Hernandez A, Scarlatti D, Costas P (2019) Real-time estimated time of arrival prediction system using historical surveillance data. In: 2019 45th Euromicro conference on software engineering and advanced applications (SEAA). Kallithea-Chalkidiki, Greece, pp 174–177

  • Pesaranghader A, Viktor H (2016) Fast hoeffding drift detection method for evolving data streams. In: 2016 Machine learning and knowledge discovery in databases (ECML-PKDD 2016). Riva del Garda, Italy, pp 96–111

  • Peterson E, Neels K, Barczi N, Graham T (2013) The economic cost of airline flight delay. J Transp Econ Policy 47(1):107–121

    Google Scholar 

  • Rebollo J, Balakrishnan H (2014) Characterization and prediction of air traffic delays. Transp Res Part C Emerg Technol 44:231–241

    Article  Google Scholar 

  • Rong F, Qianya L, Bo H, Jing Z, Dongdong Y (2015) The prediction of flight delays based the analysis of Random flight points. In: Chinese Control Conference, CCC, vol 2015-September. pp. 3992–3997

  • Schaffer C (1993) Technical note: selecting a classification method by cross-validation. Mach Learn 13(1):135–143

    Google Scholar 

  • Sternberg A, Carvalho D, Murta L, Soares J, Ogasawara E (2016) An analysis of Brazilian flight delays based on frequent patterns. Transp Res Part E Logist Transp Rev 95:282–298

    Article  Google Scholar 

  • Teixeira C, Giusti L, Soares J, Santos Jd, Amorim G, Ogasawara E (2021) Integrated dataset of Brazilian flights. In: Anais do Brazilian e-Science Workshop (BreSci). SBC, pp 89–96

  • Wang K, Li J, Tian Y (2019) Airport delay prediction method based on improved weather impacted traffic index. In: Proceedings of 2019 IEEE 1st international conference on civil aviation safety and information technology. ICCASIT 2019, pp 73–78

  • Webb G, Hyde R, Cao H, Nguyen H, Petitjean F (2016) Characterizing concept drift. Data Min Knowl Discov 30(4):964–994

    Article  MathSciNet  Google Scholar 

  • Wu CL, Law K (2019) Modelling the delay propagation effects of multiple resource connections in an airline network using a Bayesian network model. Transp Res Part E Logist Transp Rev 122:62–77

    Article  Google Scholar 

  • Yap B, Sim C (2011) Comparisons of various types of normality tests. J Stat Comput Simul 81(12):2141–2155

    Article  MathSciNet  Google Scholar 

  • Yi J, Zhang H, Liu H, Zhong G, Li G (2021) Flight delay classification prediction based on stacking algorithm. J Adv Transp 2021:1–10

    Article  Google Scholar 

  • Yu B, Guo Z, Asian S, Wang H, Chen G (2019) Flight delay prediction for commercial air transport: a deep learning approach. Transp Res Part E Logist Transp Rev 125:203–221

    Article  Google Scholar 

  • Zhou X, Angelov P (2007) Autonomous visual self-localization in completely unknown environment using evolving fuzzy rule-based classifier. In: 2007 IEEE symposium on computational intelligence in security and defense applications. Honolulu, HI, USA, pp 131–138

Download references

Acknowledgements

The authors thank CNPq, CAPES (finance code 001), and FAPERJ for partially funding this research.

Funding

The content is solely the responsibility of the authors. It does not necessarily represent the official views of the funding agencies. The funding agencies had no role in the study design, data collection, analyses, publishing decisions, or manuscript preparation.

Author information

Authors and Affiliations

  1. Federal Center for Technological Education of Rio de Janeiro, CEFET/RJ, Rio de Janeiro, Brazil

    Lucas Giusti, Leonardo Carvalho, Rafaelli Coutinho, Jorge Soares & Eduardo Ogasawara

  2. National Laboratory of Scientific Computing, LNCC, Petrópolis, Brazil

    Antonio Tadeu Gomes

Authors
  1. Lucas Giusti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonardo Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Tadeu Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rafaelli Coutinho
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jorge Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eduardo Ogasawara
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed equally to the study. EO conceptualized the study design. LG and LC acquired the data, conducted data analysis and interpretation. AG, RC, JS revised it critically for intellectual content. All authors have the approval of the final version.

Corresponding author

Correspondence to Eduardo Ogasawara.

Ethics declarations

Conflict of interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Giusti, L., Carvalho, L., Gomes, A.T. et al. Analyzing flight delay prediction under concept drift. Evolving Systems 13, 723–736 (2022). https://doi.org/10.1007/s12530-021-09415-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-021-09415-z

Keywords

Search

Navigation