Advanced Lectures on Machine Learning

Volume 2600 of the series Lecture Notes in Computer Science pp 118-183


An Introduction to Boosting and Leveraging

  • Ron MeirAffiliated withDepartment of Electrical Engineering, Technion
  • , Gunnar RätschAffiliated withResearch School of Information Sciences & Engineering The Australian National University

* Final gross prices may vary according to local VAT.

Get Access


We provide an introduction to theoretical and practical aspects of Boosting and Ensemble learning, providing a useful reference for researchers in the field of Boosting as well as for those seeking to enter this fascinating area of research. We begin with a short background concerning the necessary learning theoretical foundations of weak learners and their linear combinations. We then point out the useful connection between Boosting and the Theory of Optimization, which facilitates the understanding of Boosting and later on enables us to move on to new Boosting algorithms, applicable to a broad spectrum of problems. In order to increase the relevance of the paper to practitioners, we have added remarks, pseudo code, “tricks of the trade”, and algorithmic considerations where appropriate. Finally, we illustrate the usefulness of Boosting algorithms by giving an overview of some existing applications. The main ideas are illustrated on the problem of binary classification, although several extensions are discussed.