Classification of Binary Imbalanced Data Using A Bayesian Ensemble of Bayesian Neural Networks

  • Marcelino Lázaro
  • Francisco Herrera
  • Aníbal R. Figueiras-Vidal
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 517)


This paper presents a new method to deal with classification of imbalanced data. A Bayesian ensemble of neural network classifiers is proposed. Several individual neural classifiers are trained to minimize a Bayesian cost function with different decision costs, thus working at different points of the Receiver Operating Characteristic (ROC). Decisions of the set of individual neural classifiers are fused using a Bayesian rule that introduces a “balancing” parameter allowing to compensate the imbalance of available data.


Imbalanced data Classification Neural networks Bayes risk 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification, 2nd edn. John Wiley & Sons (2001)Google Scholar
  2. 2.
    Widrow, B., Rumelhard, D.E., Lehr, M.A.: Neural networks: Applications in industry, business and science. Communications of the ACM 37(3), 93–105 (1994)CrossRefGoogle Scholar
  3. 3.
    Zhang, G.P.: Neural networks for classification: A survey. IEEE Transactions on Systems, Man, and Cybernetics 30(4), 451–462 (2000)MathSciNetCrossRefGoogle Scholar
  4. 4.
    He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering 21(9), 1263–1284 (2009)CrossRefGoogle Scholar
  5. 5.
    Galar, M., Fernández, A., Barrenechea, E., Herrera, F.: EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling. Pattern Recognition 46, 3460–3471 (2013)CrossRefGoogle Scholar
  6. 6.
    Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics - Part C: Applications and Reviews 42(4), 463–484 (2012)CrossRefGoogle Scholar
  7. 7.
    Kay, S.M.: Fundamentals of statistical signal processing: detection theory. Prentice-Hall Inc., Upper Saddle River (1998)Google Scholar
  8. 8.
    Scharf, L.S.: Statistical signal processing: detection, estimation, and time series analysis. Addison-Wesley (1991)Google Scholar
  9. 9.
    Van Trees, H.L.: Detection, Estimation, and Modulation Theory: Part I. John Wiley and Sons, New York (1968)zbMATHGoogle Scholar
  10. 10.
    Fawcett, T.: An introduction to ROC analysis. Pattern Recognition Letters 27(8), 861–874 (2006)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7), 1145–1159 (1997)CrossRefGoogle Scholar
  12. 12.
    Cid-Sueiro, J., Arribas, J.I., Urbán-Muñoz, S., Figueiras-Vidal, A.R.: Cost functions to estimate a posteriori probabilities in multiclass problems. IEEE Transactions on Neural Networks 10(3), 645–656 (1999)CrossRefGoogle Scholar
  13. 13.
    Parzen, E.: On the estimation of a probability density function and the mode. Annals of Mathematical Statistics 33, 1065–76 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Alcalá-Fdez, A., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing 17(2–3), 255–287 (2011)Google Scholar
  15. 15.
    Rumelhart, D.E., Hinton, G.E., Willians, R.J.: Learning representations by back-propagating errors. Nature (London) 323, 533–536 (1986)CrossRefGoogle Scholar
  16. 16.
    Widrow, B., Lehr, M.A.: 30 years of adaptive neural networks: perceptron, madaline and backpropagation. Proceedings of the IEEE 78(9), 1415–1441 (1990)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Marcelino Lázaro
    • 1
  • Francisco Herrera
    • 2
  • Aníbal R. Figueiras-Vidal
    • 1
  1. 1.Department of Signal Theory and CommunicationsCarlos III University of MadridLeganésSpain
  2. 2.Department of Computer Science and Artificial IntelligenceUniversity of GranadaGranadaSpain

Personalised recommendations