Advertisement

Ensemble of Neural Networks

  • Michel Denuit
  • Donatien Hainaut
  • Julien Trufin
Chapter
Part of the Springer Actuarial book series (SPACT)

Abstract

The most frequent approach to data-driven modeling consists to estimate only a single strong predictive model. A different strategy is to build a bucket, or an ensemble of models for some particular learning task. One can consider building a set of weak or relatively weak models like small neural networks, which can be further combined altogether to produce a reliable prediction. The most prominent examples of such machine-learning ensemble techniques are random forests Breiman (Mach Learn 45:5–32, 2001) and neural network ensembles Hansen and Salamon (IEEE Trans Pattern Anal Mach Intell 12:993–1001, 1990), which have found many successful applications in different domains. Liu et al. (Earthquake prediction by RBF neural network ensemble. In: Yin F-L, Wang J, Guo C (eds) Advances in neural networks ISNN2004. Springer, Berlin, p 962–969, 2004) use this approach for predicting earthquakes. Shu and Burn (Water Resour Res 40:1–10, 2004) forecast flood frequencies with an ensemble of networks. We start this chapter by describing the bias-variance decomposition of the prediction error. Next, we discuss how aggregated models and randomized models reduce the prediction error by decreasing the variance term in the bias-variance decomposition. Theoretical developments are inspired from the PhD thesis of Louppe (Understanding random forests: from theory to practice. PhD Dissertation, Faculty of Applied Sciences, Liége University) on random forests, 2014.

References

  1. Amit Y, Geman D (1997) Shape quantization and recognition with randomized trees. Neural Comput 9(7):1545–1588CrossRefGoogle Scholar
  2. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140zbMATHGoogle Scholar
  3. Breiman L (2001) Random forests. Mach Learn 45:5–32CrossRefGoogle Scholar
  4. Geman S, Bienenstock E, Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Comput 4:1–58CrossRefGoogle Scholar
  5. Hansen L, Salamon P (1990) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12:993–1001CrossRefGoogle Scholar
  6. Liu Y, Wang Y, Li Y, Zhang B, Wu G (2004) Earthquake prediction by RBF neural network ensemble. In: Yin F-L, Wang J, Guo C (eds) Advances in neural networks ISNN2004. Springer, Berlin, pp 962–969Google Scholar
  7. Louppe G (2014) Understanding random forests: from theory to practice. PhD Dissertation, Faculty of Applied Sciences, Liége UniversityGoogle Scholar
  8. Luedtke AR, van der Laan MJ (2016) Super-learning of an optimal dynamic treatment rule. Int J Biochem 12(1):305–332MathSciNetGoogle Scholar
  9. Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ (2015) Mortality prediction in intensive care units with the super ICU learner algorithm (SICULA): a population based study. Lancet Respir Med 3(1):42–52CrossRefGoogle Scholar
  10. Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33:1–39CrossRefGoogle Scholar
  11. Shimshoni Y, Intrator N (1998) Classification of seismic signals by integrating ensembles of neural networks. IEEE Trans Signal Process 46(5):1194–1201CrossRefGoogle Scholar
  12. Shu C, Burn DH (2004) Artificial neural network ensembles and their application in pooled flood frequency analysis. Water Resour Res 40:1–10CrossRefGoogle Scholar
  13. Tin Kam H (1995) Random decision forests. In: Proceedings of the 3rd international conference on document analysis and recognition. IEEE, Piscataway, pp 278–282CrossRefGoogle Scholar
  14. Trufin J, Denuit M, Hainaut D (2019) Effective statistical learning methods for actuaries. From CART to GBM. Springer, BerlinGoogle Scholar
  15. Wolpert DH (1992) Stacked generalization. Neural Netw 5:241–259CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Michel Denuit
    • 1
  • Donatien Hainaut
    • 2
  • Julien Trufin
    • 3
  1. 1.Université Catholique LouvainLouvain-la-NeuveBelgium
  2. 2.Université Catholique de LouvainLouvain-la-NeuveFrance
  3. 3.Université Libre de BruxellesBrusselsBelgium

Personalised recommendations