Skip to main content
Log in

A hybrid recursive direct system for multi-step mortality rate forecasting

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Forecasting mortality is challenging. In general, mortality rate forecasting exercises have been based on the supposition that predictors’ residuals are random noise. However, issues regarding model selection, misspecification, or the dynamic behavior of the temporal phenomenon lead to biased or underperformed single models. Residual series might present temporal patterns that can still be used to improve the forecasting system. This paper proposes a new recursive direct multi-step Hybrid System for Mortality Forecasting (HyS-MF) that combines the Autoregressive Integrated Moving Average (ARIMA) with Neural Basis Expansion for Time Series Forecasting (N-BEATS). HyS-MF employs (i) ARIMA to model and forecast the mortality rate time series with a recursive approach and (ii) N-BEATS with the direct multi-step approach to learn and forecast the residuals of the linear predictor. The final output is generated by summing ARIMA with the N-BEATS forecasts in each time horizon. HyS-MF achieved an average Mean Absolute Percentage Error (MAPE) less than 1.34% considering all prediction horizons, beating statistical techniques, machine learning, deep learning models, and hybrid systems considering 101 different time series from the French population mortality rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

The datasets generated and analyzed during the current study are available in the “Human Mortality Database” repository, www.mortality.org.

References

  1. Bravo J (2007) Tábuas de mortalidade contemporâneas e prospectivas: Modelos estocásticos, aplicações actuariais e cobertura do risco de longevidade. PhD thesis, Universidade de Évora

  2. Shen Y, Yang X, Liu H et al (2024) Advancing mortality rate prediction in European population clusters: integrating deep learning and multiscale analysis. Sci Rep 14(1):6255

    Article  Google Scholar 

  3. van de Walk F (2017) Infant mortality and the european demographic transition. In: Watkins SC (ed) The decline of fertility in Europe. Princeton University Press, New Jersey, pp 201–233

  4. Hainaut D (2018) A neural-network analyzer for mortality forecast. ASTIN Bull: J IAA 48(2):481–508

    Article  MathSciNet  Google Scholar 

  5. Kruk ME, Gage AD, Joseph NT et al (2018) Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. The Lancet 392(10160):2203–2212

    Article  Google Scholar 

  6. Chen N, Pan J (2022) The causal effect of delivery volume on severe maternal morbidity: an instrumental variable analysis in sichuan, china. BMJ Glob Health 7(5):e008428

    Article  Google Scholar 

  7. Luy M, Di Giulio P, Di Lego V et al (2020) Life expectancy: frequently used, but hardly understood. Gerontology 66(1):95–104

    Article  Google Scholar 

  8. Bravo JM (2021) Forecasting mortality rates with recurrent neural networks: a preliminary investigation using portuguese data. In: CAPSI 2021 Proceedings

  9. Nigri A, Levantesi S, Aburto JM (2022) Leveraging deep neural networks to estimate age-specific mortality from life expectancy at birth. Demogr Res 47:199–232

    Article  Google Scholar 

  10. Carone G, Eckefeldt P, Giamboni L, et al (2016) Pension reforms in the EU since the early 2000’s: achievements and challenges ahead. European economy discussion paper

  11. Janssen F (2018) Advances in mortality forecasting: introduction. Genus 74(1):21

    Article  Google Scholar 

  12. Olivieri A (2001) Uncertainty in mortality projections: an actuarial perspective. Insur: Math Econ 29(2):231–245

    Google Scholar 

  13. Shi Y (2021) Forecasting mortality rates with the penalized exponential smoothing state space model. J Operat Res Soci 73(5):955–968

    Article  Google Scholar 

  14. Hyndman RJ, Shahid Ullah M (2007) Robust forecasting of mortality and fertility rates: a functional data approach. Comput Stat Data Anal 51(10):4942–4956

    Article  MathSciNet  Google Scholar 

  15. Vanella P, Deschermeier P, Wilke CB (2020) An overview of population projections-methodological concepts, international data availability, and use cases. Forecasting 2(3):346–363

    Article  Google Scholar 

  16. Dushi I, Friedberg L, Webb T (2010) The impact of aggregate mortality risk on defined benefit pension plans. J Pension Econ Finance 9(4):481–503

    Article  Google Scholar 

  17. Mitchell D, Brockett P, Mendoza-Arriaga R et al (2013) Modeling and forecasting mortality rates. Insur: Math Econ 52(2):275–285

    MathSciNet  Google Scholar 

  18. Wang J, Wen L, Xiao L et al (2024) Time-series forecasting of mortality rates using transformer. Scand Actuar J 2:109–123

    Article  MathSciNet  Google Scholar 

  19. Bi L, Fili M, Hu G (2022) Covid-19 forecasting and intervention planning using gated recurrent unit and evolutionary algorithm. Neural Computing and Applications pp 1–19

  20. Wu R, Wang B (2018) Gaussian process regression method for forecasting of mortality rates. Neurocomputing 316:232–239

    Article  Google Scholar 

  21. Feng L, Shi Y (2018) Forecasting mortality rates: multivariate or univariate models? J Popul Res 35(3):289–318

    Article  Google Scholar 

  22. Lee RD, Carter LC (1992) Modeling and forecasting US mortality. J Am Stat Associat 87(419):659–671

    Google Scholar 

  23. Booth H, Maindonald J, Smith L (2002) Applying Lee-Carter under conditions of variable mortality decline. Popul Stud 56(3):325–336

    Article  Google Scholar 

  24. Deprez P, Shevchenko PV, Wüthrich MV (2017) Machine learning techniques for mortality modeling. Eur Actuar J 7(2):337–352

    Article  MathSciNet  Google Scholar 

  25. Nigri A, Levantesi S, Marino M et al (2019) A deep learning integrated lee-carter model. Risks 7(1):33

    Article  Google Scholar 

  26. McNown R, Rogers A (1989) Forecasting mortality: a parameterized time series approach. Demography 26(4):645–660

    Article  Google Scholar 

  27. de Mattos Neto PS, Cavalcanti GD, Madeiro F (2017) Nonlinear combination method of forecasters applied to PM time series. Patt Recogn Lett 95:65–72

    Article  Google Scholar 

  28. Richman R, Wüthrich MV (2018) A neural network extension of the Lee-Carter Model to multiple populations. SSRN

  29. Petneházi G, Gáll J (2019) Mortality rate forecasting: can recurrent neural networks beat the lee-carter model? arXiv preprint arXiv:1909.05501

  30. Perla F, Richman R, Scognamiglio S et al (2021) Time-series forecasting of mortality rates using deep learning. Scand Actuar J 2021(7):572–598

    Article  MathSciNet  Google Scholar 

  31. Hong WH, Yap JH, Selvachandran G et al (2021) Forecasting mortality rates using hybrid lee-carter model, artificial neural network and random forest. Complex Intell Syst 7:163–189

    Article  Google Scholar 

  32. Chen Y, Khaliq AQ (2022) Comparative study of mortality rate prediction using data-driven recurrent neural networks and the lee-carter model. Big Data Cognit Comput 6(4):134

    Article  Google Scholar 

  33. Roshani A, Izadi M, Khaledi BE (2022) Transformer self-attention network for forecasting mortality rates. J Iran Stat Soci 21(1):81–103

    MathSciNet  Google Scholar 

  34. de Mattos Neto PS, Cavalcanti GD, de Santos Júnior ODS et al (2022) Hybrid systems using residual modeling for sea surface temperature forecasting. Scientific Reports 12(1):487

    Article  Google Scholar 

  35. Pang X, Zhou Y, Wang P et al (2020) An innovative neural network approach for stock market prediction. J Supercomput 76:2098–2118

    Article  Google Scholar 

  36. Bravo JM (2021) Forecasting longevity for financial applications: a first experiment with deep learning methods. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, pp 232–249

  37. Bravo JM, Santos V (2021) Backtesting recurrent neural networks with gated recurrent unit: probing with chilean mortality data. In: International Conference on Computer Science. Springer, Electronics and Industrial Engineering (CSEI), pp 159–174

  38. Jackins V, Vimal S, Kaliappan M et al (2021) Ai-based smart prediction of clinical disease using random forest classifier and naive bayes. J Supercomput 77(5):5198–5219

    Article  Google Scholar 

  39. Ashofteh A, Bravo JM, Ayuso M (2022) An ensemble learning strategy for panel time series forecasting of excess mortality during the covid-19 pandemic. Appl Soft Comput 128:109422

    Article  Google Scholar 

  40. Xu Y, Wang E, Yang Y et al (2021) A unified collaborative representation learning for neural-network based recommender systems. IEEE Trans Knowl Data Eng 34(11):5126–5139

    Article  Google Scholar 

  41. Kavianpour P, Kavianpour M, Jahani E et al (2023) A cnn-bilstm model with attention mechanism for earthquake prediction. J Supercomput 79(17):19194–19226

    Article  Google Scholar 

  42. Santos WR, Sampaio AR Jr, Rosa NS et al (2024) Microservices performance forecast using dynamic multiple predictor systems. Eng Appl Artif Intell 129:107649

    Article  Google Scholar 

  43. Zhang G (2003) Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50:159–175

    Article  Google Scholar 

  44. Khashei M, Bijari M (2011) A novel hybridization of artificial neural networks and ARIMA models for time series forecasting. Appl Soft Comput 11(2):2664–2675

    Article  Google Scholar 

  45. Meng H, Han L, Hou L (2022) An ensemble learning-based short-term load forecasting on small datasets. In: 2022 IEEE 33rd Annual International Symposium on Personal. Indoor and Mobile Radio Communications (PIMRC), IEEE, pp 346–350

  46. de Mattos Neto PS, de Oliveira JF, de O Santos Júnior DS, et al (2021) An adaptive hybrid system using deep learning for wind speed forecasting. Inform Sci 581:495–514

  47. Olson M, Wyner A, Berk R (2018) Modern neural networks generalize on small data sets. In: Bengio S, Wallach H, Larochelle H, et al (eds) Advances in Neural Information Processing Systems, vol 31. Curran Associates, Inc., https://proceedings.neurips.cc/paper_files/paper/2018/file/fface8385abbf94b4593a0ed53a0c70f-Paper.pdf

  48. Shaikhina T, Lowe D, Daga S et al (2015) Machine learning for predictive modelling based on small data in biomedical engineering. IFAC-PapersOnLine 48(20):469–474

    Article  Google Scholar 

  49. D’souza RN, Huang PY, Yeh FC (2020) Structural analysis and optimization of convolutional neural networks with a small sample size. Sci Rep 10(1):834

    Article  Google Scholar 

  50. Meroni M, Waldner F, Seguini L et al (2021) Yield forecasting with machine learning and small data: what gains for grains? Agric For Meteorol 308:108555

    Article  Google Scholar 

  51. Oreshkin BN, Carpov D, Chapados N, et al. (2019) N-beats: neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437

  52. Oreshkin BN, Dudek G, Pełka P et al (2021) N-beats neural network for mid-term electricity load forecasting. Appl Energy 293:116918

    Article  Google Scholar 

  53. Human Mortality Database (2021) University of California, Berkeley (USA), and Max Plank Institute for Demographic Research (Germany). Available at www.mortality.org; accessed on 04/20/2021

  54. Hyndman RJ, Koehler AB, Snyder RD et al (2002) A state space framework for automatic forecasting using exponential smoothing methods. Int J Forecast 18(3):439–454

    Article  Google Scholar 

  55. De Livera AM, Hyndman RJ, Snyder RD (2011) Forecasting time series with complex seasonal patterns using exponential smoothing. J Am Stat Assoc 106(496):1513–1527

    Article  MathSciNet  Google Scholar 

  56. Box GE, Jenkins GM, Reinsel GC et al (2015) Time series analysis: forecasting and control. Wiley, Hoboken

    Google Scholar 

  57. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. Advances in neural information processing systems 30

  58. Wu H, Xu J, Wang J et al (2021) Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv Neural Inf Process Syst 34:22419–22430

    Google Scholar 

  59. Zhou T, Ma Z, Wen Q, et al. (2022) Fedformer: fenhanced decomposed transformer for long-term series forecasting. In: International Conference on Machine Learning, PMLR, pp 27268–27286

  60. Challu C, Olivares KG, Oreshkin BN, et al. (2023) Nhits: neural hierarchical interpolation for time series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 6989–6997

  61. Bell WR (1997) Comparing and assessing time series methods for forecasting age-specific fertility and mortality rates. J Off Stat 13:279–303

    Google Scholar 

  62. Renshaw A, Haberman S (2006) A cohort-based extension to the Lee-Carter model for mortality reduction factors. Insur: Math Econ 38(3):556–570

    Google Scholar 

  63. Hyndman RJ, Booth H, Yasmeen F (2013) Coherent mortality forecasting: the product-ratio method with functional time series models. Demography 50(1):261–283

    Article  Google Scholar 

  64. Wu R, Wang B (2019) Coherent mortality forecasting by the weighted multilevel functional principal component approach. J Appl Stat 46(10):1774–1791

    Article  MathSciNet  Google Scholar 

  65. Richmond P, Roehner BM, Irannezhad A et al (2021) Mortality: a physics perspective. Physica A 566:125660

    Article  MathSciNet  Google Scholar 

  66. Makridakis S, Hyndman RJ, Petropoulos F (2020) Forecasting in social settings: the state of the art. Int J Forecast 36(1):15–28

    Article  Google Scholar 

  67. Booth H, Tickle L (2008) Mortality modelling and forecasting: a review of methods. Ann Act Sci 3(1–2):3–43

    Article  Google Scholar 

  68. Giacometti R, Bertocchi M, Rachev ST et al (2012) A comparison of the Lee-Carter model and AR-ARCH model for forecasting mortality rates. Insur: Math Econ 50(1):85–93

    MathSciNet  Google Scholar 

  69. Shang HL, Hyndman RJ (2017) Grouped functional time series forecasting: an application to age-specific mortality rates. J Comput Graph Stat 26(2):330–343

    Article  MathSciNet  Google Scholar 

  70. Santos JDSdO, Oliveira JFd, de Mattos Neto PSG (2019) An intelligent hybridization of ARIMA with machine learning models for time series forecasting. Knowl-Based Syst 175:72–86

    Article  Google Scholar 

  71. Hajirahimi Z, Khashei M (2019) Hybrid structures in time series modeling and forecasting: A review. Eng Appl Artif Intell 86:83–106

    Article  Google Scholar 

  72. Pai PF, Lin CS (2005) A hybrid ARIMA and support vector machines model in stock price forecasting. Omega 33(6):497–505

    Article  Google Scholar 

  73. Panigrahi S, Behera HS (2017) A hybrid ETS-ANN model for time series forecasting. Eng Appl Artif Intell 66:49–59

    Article  Google Scholar 

  74. Hajirahimi Z, Khashei M (2019) Weighted sequential hybrid approaches for time series forecasting. Physica A: Stat Mech Appl 531

  75. Babu CN, Reddy BE (2014) A moving-average filter based hybrid arima-ann model for forecasting time series data. Appl Soft Comput 23:27–38

    Article  Google Scholar 

  76. Shi J, Guo J, Zheng S (2012) Evaluation of hybrid forecasting approaches for wind speed and power generation time series. Renew Sustain Energy Rev 16(5):3471–3480

    Article  Google Scholar 

  77. Chakraborty T, Chattopadhyay S, Ghosh I (2019) Forecasting dengue epidemics using a hybrid methodology. Physica A 527:121266

    Article  Google Scholar 

  78. Iftikhar H, Daniyal M, Qureshi M et al (2023) A hybrid forecasting technique for infection and death from the mpox virus. Digital Health 9:20552076231204748

    Article  Google Scholar 

  79. Iftikhar H, Zafar A, Turpo-Chaparro JE et al (2023) Forecasting day-ahead brent crude oil prices using hybrid combinations of time series models. Mathematics 11(16):3548

    Article  Google Scholar 

  80. Carbo-Bustinza N, Iftikhar H, Belmonte M et al (2023) Short-term forecasting of ozone concentration in metropolitan lima using hybrid combinations of time series models. Appl Sci 13(18):10514

    Article  Google Scholar 

  81. Sorjamaa A, Hao J, Reyhani N et al (2007) Methodology for long-term prediction of time series. Neurocomputing 70(16–18):2861–2869

    Article  Google Scholar 

  82. Hamzaçebi C, Akay D, Kutay F (2009) Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting. Expert Syst Appl 36(2):3839–3844

    Article  Google Scholar 

  83. Taieb SB, Bontempi G, Atiya AF et al (2012) A review and comparison of strategies for multi-step ahead time series forecasting based on the nn5 forecasting competition. Expert Syst Appl 39(8):7067–7083

    Article  Google Scholar 

  84. Kline DM (2004) Methods for multi-step time series forecasting neural networks. In: Zhang GP (ed) Neural networks in business forecasting. IGI Global, Hershey, PA, USA, pp 226–250

  85. Bontempi G (2008) Long term time series prediction with multi-input multi-output local learning. In: Proceedings of the 2nd European Symposium on Time Series Prediction (TSP), ESTSP08

  86. Ming W, Bao Y, Hu Z, et al (2014) Multistep-ahead air passengers traffic prediction with hybrid arima-svms models. The Scientific World Journal 2014

  87. Taieb SB, Bontempi G, Sorjamaa A, et al. (2009) Long-term prediction of time series by combining direct and mimo strategies. In: 2009 International Joint Conference on Neural Networks, IEEE, pp 3054–3061

  88. Beyaztas U, Shang H (2022) Machine-learning-based functional time series forecasting: application to age-specific mortality rates. Forecasting 4(1):394–408

    Article  Google Scholar 

  89. Ouyang Z, Ravier P, Jabloun M (2022) Are deep learning models practically good as promised? a strategic comparison of deep learning models for time series forecasting. In: 2022 30th European Signal Processing Conference (EUSIPCO), IEEE, pp 1477–1481

  90. Atiya A, El-Shoura S, Shaheen S et al (1999) A comparison between neural-network forecasting techniques-case study: river flow forecasting. IEEE Trans Neural Networks 10(2):402–409

    Article  Google Scholar 

  91. Taieb SB (2014) Machine learning strategies for multi-step-ahead time series forecasting. Universit Libre de Bruxelles, Belgium pp 75–86

  92. Mendes-Moreira J, Soares C, Jorge AM et al (2012) Ensemble approaches for regression: a survey. ACM Comput Surv (csur) 45(1):1–40

    Article  Google Scholar 

  93. Lam KK, Wang B (2021) Robust non-parametric mortality and fertility modelling and forecasting: Gaussian process regression approaches. Forecasting 3(1):207–227

    Article  Google Scholar 

  94. da Rocha AM, Espíndola AL, Penna T (2020) Mortality curves using a bit-string aging model. Physica A 560:125134

    Article  Google Scholar 

  95. Hyndman RJ, Booth H (2008) Stochastic population forecasts using functional data models for mortality, fertility and migration. Int J Forecast 24(3):323–342

    Article  Google Scholar 

  96. Chandra R, Goyal S, Gupta R (2021) Evaluation of deep learning models for multi-step ahead time series prediction. IEEE Access 9:83105–83123

    Article  Google Scholar 

  97. Smith TG, et al. (2017–) Pmdarima: Arima estimators for Python. http://www.alkaline-ml.com/pmdarima

  98. Hyndman RJ, Khandakar Y (2008) Automatic time series forecasting: the forecast package for R. J Stat Softw 27(1):1–22

    Google Scholar 

  99. Hyndman R, Athanasopoulos G, Bergmeir C, et al. (2024) Forecast: forecasting functions for time series and linear models. https://pkg.robjhyndman.com/forecast/, r package version 8.22.0

  100. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  Google Scholar 

  101. Chollet F, et al. (2015) Keras. https://keras.io

  102. Abadi M, Agarwal A, Barham P, et al. (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/, software available from tensorflow.org

  103. Olivares KG, Challú C, Garza F, et al. (2022) NeuralForecast: user friendly state-of-the-art neural forecasting models. PyCon Salt Lake City, Utah, US 2022, https://github.com/Nixtla/neuralforecast

  104. Paszke A, Gross S, Massa F, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems 32. Curran Associates, Inc., p 8024–8035, http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

  105. de Oliveira JF, Silva EG, de Mattos Neto PS (2021) A hybrid system based on dynamic selection for time series forecasting. IEEE Trans Neural Netw Learn Syst 33(8):3251–3263

    Article  MathSciNet  Google Scholar 

  106. Kim S, Kim H (2016) A new metric of absolute percentage error for intermittent demand forecasts. Int J Forecast 32(3):669–679

    Article  Google Scholar 

  107. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688

    Article  Google Scholar 

  108. Diebold FX, Mariano RS (1995) Comparing predictive accuracy. J Bus Econ Stat 13(3):253–263

    Article  Google Scholar 

  109. Xu C, Xie Y (2023) Conformal prediction for time series. IEEE Trans Patt Anal Mach Intell. https://doi.org/10.1109/TPAMI.2023.3272339

    Article  Google Scholar 

  110. Medina MCC, de Oliveira JFL (2023) A selective hybrid system for state-of-charge forecasting of lithium-ion batteries. J Supercomput 79(14):15623–15642

    Article  Google Scholar 

  111. Silva EG, Júunior DSdO, Cavalcanti GD, et al. (2018) Improving the accuracy of intelligent forecasting models using the perturbation theory. In: 2018 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1–7

Download references

Funding

No funding.

Author information

Authors and Affiliations

Authors

Contributions

Filipe Coelho de Lima Duarte contributed to conceptualization, methodology, software, investigation, writing—original draft, and writing—review & editing. Paulo Salgado Gomes de Mattos Neto contributed to supervision, and writing—review & editing. Paulo Renato Alves Firmino contributed to writing—review & editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Filipe Coelho de Lima Duarte.

Ethics declarations

Conflict of interest

The authors that they have no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Lima Duarte, F.C., de Mattos Neto, P.S.G. & Firmino, P.R.A. A hybrid recursive direct system for multi-step mortality rate forecasting. J Supercomput (2024). https://doi.org/10.1007/s11227-024-06182-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06182-x

Keywords

Navigation