Advertisement

Improving the Weighted Distribution Estimation for AdaBoost Using a Novel Concurrent Approach

  • Héctor Allende-CidEmail author
  • Carlos Valle
  • Claudio Moraga
  • Héctor Allende
  • Rodrigo Salas
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 616)

Abstract

AdaBoost is one of the most known Ensemble approaches used in the Machine Learning literature. Several AdaBoost approaches that use Parallel processing, in order to speed up the computation in Large datasets, have been recently proposed. These approaches try to approximate the classic AdaBoost, thus sacrificing its generalization ability. In this work, we use Concurrent Computing in order to improve the Distribution Weight estimation, hence obtaining improvements in the capacity of generalization. We train in parallel in each round several weak hypotheses, and using a weighted ensemble we update the distribution weights of the following boosting rounds. Our results show that in most cases the performance of AdaBoost is improved and that the algorithm converges rapidly. We validate our proposal with 4 well-known real data sets.

Notes

Acknowledgments

This work was supported by the following research grants: Fondecyt 1110854 and DGIP-UTFSM. The work of C. Moraga was partially supported by the Foundation for the Advancement of Soft Computing, Mieres, Spain.

References

  1. 1.
    Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Mach. Learn. 36(1–2), 105–139 (1999)CrossRefGoogle Scholar
  2. 2.
    Clemen, R.T.: Combining forecasts: a review and annotated bibliography. Int. J. Forecast. 5(4), 559–583 (1989)CrossRefGoogle Scholar
  3. 3.
    Fercoq, O.: Parallel Coordinate Descent for the Adaboost Problem. In: Proceedings of the 12th International Conference on Machine Learning and Applications (ICMLA), vol. 1, no. 1, pp. 354-358, 4-7 December 2013Google Scholar
  4. 4.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Freund, Y., Shapire, R.E.: A short introduction to boosting. J. Jpn. Soc. Artif. Int. 14(5), 771–780 (1999)Google Scholar
  6. 6.
    Kuncheva, L., Whitaker, C.: Using diversity with three variants of boosting: aggressive, conservative and inverse. Lect. Notes Comput. Sci. 2364(1), 81–90 (2002)CrossRefzbMATHGoogle Scholar
  7. 7.
    Kyrkou, C., Theocharides, T.: A flexible parallel hardware architecture for adaboost-based real-time object detection. IEEE Trans. Very Larg. Scale Integr. (VLSI) Syst. 19(6), 1034–1047 (2011)CrossRefGoogle Scholar
  8. 8.
    Lichman, M.: UCI Machine Learning Repository. University of California, Irvine. School of Information and Computer Science (2013). http://archive.ics.uci.edu/ml
  9. 9.
    Liu, H., Tian, H.Q., Li, Y.F., Zhang, L.: Comparison of four Adaboost algorithm based artificial neural networks in wind speed predictions. Energy Convers. Manage. 92(1), 67–81 (2015)CrossRefGoogle Scholar
  10. 10.
    Mukherjee, I., Canini, K., Frongillo, R., Singer, Y.: Parallel boosting with momentum. Mach. Learn. Knowl. Discov. Databases 8190(1), 17–32 (2013)CrossRefGoogle Scholar
  11. 11.
    Palit, I., Reddy, C.K.: Parallelized Boosting with Map-Reduce. In: IEEE International Conference on Data Mining Workshops (ICDMW), vol. 1, no. 1, pp. 1346–1353, 13 December 2010Google Scholar
  12. 12.
    Palit, I., Reddy, C.K.: Scalable and parallel boosting with mapreduce. IEEE Trans. Knowl. Data Eng. 24(10), 1904–1916 (2012)CrossRefGoogle Scholar
  13. 13.
    Seiffert, C., Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Resampling or reweighting: a comparison of boosting implementations. In Proceedings of the 20th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 08), vol. 1, no. 1, pp. 445–451, November 2008Google Scholar
  14. 14.
    Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Héctor Allende-Cid
    • 1
    Email author
  • Carlos Valle
    • 2
  • Claudio Moraga
    • 4
    • 5
  • Héctor Allende
    • 2
  • Rodrigo Salas
    • 3
  1. 1.Escuela de Ingeniería InformáticaPontificia Universidad Católica de ValparaísoValpara ísoChile
  2. 2.Departamento de InformáticaUniversidad Técnica Federico Santa MaríaValpara ísoChile
  3. 3.Escuela de Ingeniería BiomédicaUniversidad de ValparaísoValpara ísoChile
  4. 4.European Centre for Soft ComputingAsturiasSpain
  5. 5.TU Dortmund UniversityDortmundGermany

Personalised recommendations