Skip to main content
SpringerLink
Log in

Ensemble Learning

  • Chapter
  • First Online:
Machine Learning

Abstract

Ensemble learning, also known as multiple classifier system and committee-based learning, trains and combines multiple learners to solve a learning problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 64.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Breiman L (1996a) Bagging predictors. Mach Learn 24(2):123–140

    Google Scholar 

  • Breiman L (1996b) Stacked regressions. Mach Learn 24(1):49−64

    Google Scholar 

  • Breiman L (2000) Randomizing outputs to increase prediction accuracy. Mach Learn 40(3):113−120

    Google Scholar 

  • Breiman L (2001a) Random forests. Mach Learn 45(1):5−32

    Google Scholar 

  • Breiman L (2001b) Using iterated bagging to debias regressions. Mach Learn 45(3):261−277

    Google Scholar 

  • Clarke B (2003) Comparing Bayes model averaging and stacking when model approximation error cannot be ignored. J Mach Learn Res 4:683−712

    Google Scholar 

  • Demiriz A, Bennett KP, Shawe-Taylor J (2008) Linear programming Boosting via column generation. Mach Learn 46(1−3):225−254

    Google Scholar 

  • Dietterich TG(2000) Ensemble methods in machine learning. In: Proceedings of the 1st international workshop on multiple classifier systems (MCS), pp 1−15. Cagliari, Italy

    Google Scholar 

  • Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263−286

    Google Scholar 

  • Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119−139

    Google Scholar 

  • Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting (with discussions). Ann Stat 28(2):337−407

    Google Scholar 

  • Friedman JH (2001) Greedy function approximation: a gradient Boosting machine. Ann Stat 29(5):1189−1232

    Google Scholar 

  • Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Patt Anal Mach Intell 20(8):832−844

    Google Scholar 

  • Ho TK, Hull JJ, Srihari SN (1994) Decision combination in multiple classifier systems. IEEE Trans Patt Anal Mach Intell 16(1):66−75

    Google Scholar 

  • Kearns M, Valiant LG (1989) Cryptographic limitations on learning Boolean formulae and finite automata. In: Proceedings of the 21st annual ACM symposium on theory of computing (STOC), pp 433−444. Seattle, WA

    Google Scholar 

  • Kittler J, Hatef M, Duin R, Matas J (1998) On combining classifiers. IEEE Trans Patt Anal Mach Intell 20(3):226−239

    Google Scholar 

  • Kohavi R, Wolpert DH (1996) Bias plus variance decomposition for zeroone loss functions. In: Proceedings of the 13th international conference on machine learning (ICML), pp 275−283. Bari, Italy

    Google Scholar 

  • Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation and active learning. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems 7 (NIPS). MIT Press, Cambridge, MA, pp 231−238

    Google Scholar 

  • Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley, Hoboken, NJ

    Google Scholar 

  • Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2):181−207

    Google Scholar 

  • Liu Y, Yao X (1999) Ensemble learning via negative correlation. Neural Netw 12(10):1399−1404

    Google Scholar 

  • Markowitz H (1952) Portfolio selection. J Financ 7(1):77−91

    Google Scholar 

  • Mease D, Wyner A (2008) Evidence contrary to the statistical view of boosting (with discussions). J Mach Learn Res 9:131−201

    Google Scholar 

  • Perrone MP, Cooper LN (1993) When networks disagree: ensemble method for neural networks. In: Mammone RJ (ed) Artificial neural networks for speech and vision. Chapman & Hall, New York, NY, pp 126−142

    Google Scholar 

  • Platt J (2000) Probabilities for SV machines. In: Smola A, Bartlett P, Schölkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, MA, pp 61−74

    Google Scholar 

  • Rokach L (2010a) Ensemble-based classifiers. Artif Intell Rev 33(1):1−39

    Google Scholar 

  • Rokach L (2010b) Pattern classification using ensemble methods. World Scientific, Singapore

    Google Scholar 

  • Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197−227

    Google Scholar 

  • Schapire RE, Freund Y (2012) Boosting: foundations and algorithms. MIT Press, Cambridge, MA

    Google Scholar 

  • Schapire RE, Freund Y, Bartlett P, Lee WS (1998) Boosting the margin: a new explanation for the effectiveness of voting methods. Ann Stat 26(5):1651−1686

    Google Scholar 

  • Seewald AK (2002) How to make stacking better and faster while also taking care of an unknown weakness. In: Proceedings of the 19th international conference on machine learning (ICML), pp 554−561. Sydney, Australia

    Google Scholar 

  • Tang EK, Suganthan PN, YaoX(2006) An analysis of diversity measures. Mach Learn 65(1):247−271

    Google Scholar 

  • Ting KM, Witten IH (1999) Issues in stacked generalization. J Artif Intell Res 10:271−289

    Google Scholar 

  • Webb GI (2000) MultiBoosting: a technique for combining boosting and wagging. Mach Learn 40(2):159−196

    Google Scholar 

  • Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241−260

    Google Scholar 

  • Wolpert DH, Macready WG (1999) An efficient method to estimate Bagging’s generalization error. Mach Learn 35(1):41−55

    Google Scholar 

  • Xu L, Krzyzak A, Suen CY (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybern 22(3):418−435

    Google Scholar 

  • Zadrozny B, Elkan C (2001) Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. In: Proceedings of the 18th international conference on machine learning (ICML), pp 609−616. Williamstown, MA

    Google Scholar 

  • Zhou Z-H (2012) Ensemble methods: foundations and algorithms. Chapman & Hall/CRC, Boca Raton, FL

    Google Scholar 

  • Zhou Z-H (2014) Large margin distribution learning. In: Proceedings of the 6th IAPR international workshop on artificial neural networks in pattern recognition (ANNPR), pp 1−11. Montreal, Canada

    Google Scholar 

  • Zhou Z-H, Jiang Y (2004) NeC4.5: neural ensemble based C4.5. IEEE Trans Knowl Data Eng 16(6):770−773

    Google Scholar 

  • Zhou Z-H, Wu J, Wei T (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1−2):239−263

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanjing University, Nanjing, Jiangsu, China

    Zhi-Hua Zhou

Authors
  1. Zhi-Hua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hua Zhou .

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhou, ZH. (2021). Ensemble Learning. In: Machine Learning. Springer, Singapore. https://doi.org/10.1007/978-981-15-1967-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-1967-3_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-1966-6

  • Online ISBN: 978-981-15-1967-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation