A Preliminary Study of Diversity in Extreme Learning Machines Ensembles

  • Carlos Perales-GonzálezEmail author
  • Mariano Carbonero-Ruz
  • David Becerra-Alonso
  • Francisco Fernández-Navarro
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10870)


In this paper, the neural network version of Extreme Learning Machine (ELM) is used as a base learner for an ensemble meta-algorithm which promotes diversity explicitly in the ELM loss function. The cost function proposed encourages orthogonality (scalar product) in the parameter space. Other ensemble-based meta-algorithms from AdaBoost family are used for comparison purposes. Both accuracy and diversity presented in our proposal are competitive, thus reinforcing the idea of introducing diversity explicitly.


Extreme learning machine Diversity Machine learning Ensemble AdaBoost 


  1. 1.
    Witten, I.H., Frank, E., Hall, M.A., Pal, C.J.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2016)Google Scholar
  2. 2.
    Mohri, M., Rostamizadeh, A., Talwalkar, A.: Foundations of Machine Learning. MIT press, Cambridge (2012)zbMATHGoogle Scholar
  3. 3.
    Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). Scholar
  4. 4.
    Zhang, L., Shah, S., Kakadiaris, I.: Hierarchical multi-label classification using fully associative ensemble learning. Pattern Recogn. 70, 89–103 (2017)CrossRefGoogle Scholar
  5. 5.
    Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Woźniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)CrossRefGoogle Scholar
  6. 6.
    Polikar, R.: Ensemble learning. In: Zhang, C., Ma, Y. (eds.) Ensemble Machine Learning, pp. 1–34. Springer, Boston (2012). Scholar
  7. 7.
    Bbeiman, L.: Bagging predictors. Mach, Learn. 24, 123–140 (1996)zbMATHGoogle Scholar
  8. 8.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)CrossRefGoogle Scholar
  10. 10.
    Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(2), 513–29 (2012)CrossRefGoogle Scholar
  11. 11.
    Tian, H., Meng, B.: A new modeling method based on bagging elm for day-ahead electricity price prediction. In: 2010 IEEE Fifth International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA), pp. 1076–1079. IEEE (2010)Google Scholar
  12. 12.
    Riccardi, A., Fernández-Navarro, F., Carloni, S.: Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine. IEEE Trans. Cybern. 44(10), 1898–1909 (2014)CrossRefGoogle Scholar
  13. 13.
    Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)CrossRefGoogle Scholar
  14. 14.
    Freund, Y., Schapire, R.E.: A short introduction to boosting. J. Jpn. Soc. Artif. Intell. 14(5), 771–780 (1999)Google Scholar
  15. 15.
    Lichman, M.: UCI machine learning repository (2013)Google Scholar
  16. 16.
    Ran, Y., Sun, X., Sun, H., Sun, L., Wang, X., Ran, Y.W.X., Sun, X., Sun, H., Sun, L.: Boosting ridge extreme learning machine. In: Proceedings - 2012 IEEE Symposium on Robotics and Applications, ISRA 2012, pp. 881–884 (2012)Google Scholar
  17. 17.
    Wang, S., Chen, H., Yao, X.: Negative correlation learning for classification ensembles. In: Proceedings of the International Joint Conference on Neural Networks (2010)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Carlos Perales-González
    • 1
    Email author
  • Mariano Carbonero-Ruz
    • 1
  • David Becerra-Alonso
    • 1
  • Francisco Fernández-Navarro
    • 1
  1. 1.Department of Quantitative MethodsUniversidad Loyola AndaluciaSevillaSpain

Personalised recommendations