Bayesian Chain Classifier with Feature Selection for Multi-label Classification

  • Ricardo Benítez JiménezEmail author
  • Eduardo F. Morales
  • Hugo Jair Escalante
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11288)


Multi-label classification task has many applications in Text Categorization, Multimedia, Biology, Chemical data analysis and Social Network Mining, among others. Different approaches have been developed: Binary Relevance (BR), Label Power Set (LPS), Random k label sets (RAkEL), some of them consider the interaction between labels in a chain (Chain Classifier) and other alternatives around this method are derived, for instance, Probabilistic Chain Classifier, Monte Carlo Chain Classifier and Bayesian Chain Classifier (BCC). All previous approaches have in common and focus on is in considering different orders or combinations of the way the labels have to be predicted. Given that feature selection has proved to be important in classification tasks, reducing the dimensionality of the problem and even improving classification model’s accuracy. In this work a feature selection technique is tested in BCC algorithm with two searching methods, one using Best First (BF-FS-BCC) and another with GreedyStepwise (GS-FS-BCC), these methods are compared, the winner is also compared with BCC, both tests are compared through Wilcoxon Signed Rank test, in addition it is compared with others Chain Classifier and finally it is compared with others approaches (BR, RAkEL, LPS).


Multi-label classification Chain classifier BCC Feature selection 


  1. 1.
    Dembczynski, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: ICML, vol. 10, pp. 279–286 (2010)Google Scholar
  2. 2.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7(Jan), 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Hall, M.A.: Correlation-based feature selection for machine learning (1999)Google Scholar
  4. 4.
    Lastra, G., Luaces, O., Quevedo, J.R., Bahamonde, A.: Graphical feature selection for multilabel classification tasks. In: Gama, J., Bradley, E., Hollmén, J. (eds.) IDA 2011. LNCS, vol. 7014, pp. 246–257. Springer, Heidelberg (2011). Scholar
  5. 5.
    Read, J., Martino, L., Luengo, D.: Efficient Monte Carlo optimization for multi-label classifier chains. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3457–3461. IEEE (2013)Google Scholar
  6. 6.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85(3), 333 (2011)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Read, J., Reutemann, P., Pfahringer, B., Holmes, G.: MEKA: a multi-label/multi-target extension to WEKA. J. Mach. Learn. Res. 17(21), 1–5 (2016).
  8. 8.
    Spolaôr, N., Cherman, E.A., Monard, M.C., Lee, H.D.: Filter approach feature selection methods to support multi-label learning based on relieff and information gain. In: Barros, L.N., Finger, M., Pozo, A.T., Gimenénez-Lugo, G.A., Castilho, M. (eds.) SBIA 2012. LNCS (LNAI), pp. 72–81. Springer, Heidelberg (2012). Scholar
  9. 9.
    SpolaôR, N., Cherman, E.A., Monard, M.C., Lee, H.D.: A comparison of multi-label feature selection methods using the problem transformation approach. Electron. Notes Theor. Comput. Sci. 292, 135–151 (2013)CrossRefGoogle Scholar
  10. 10.
    Sucar, L.E., Bielza, C., Morales, E.F., Hernandez-Leal, P., Zaragoza, J.H., Larrañaga, P.: Multi-label classification with Bayesian network-based chain classifiers. Pattern Recogn. Lett. 41, 14–22 (2014)CrossRefGoogle Scholar
  11. 11.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Random k-labelsets for multilabel classification. IEEE Trans. Knowl. Data Eng. 23(7), 1079–1089 (2011)CrossRefGoogle Scholar
  12. 12.
    Tsoumakas, G., Katakis, I., et al.: Multi-label classification: an overview. Int. J. Data Warehous. Min. (IJDWM) 3(3), 1–13 (2007)CrossRefGoogle Scholar
  13. 13.
    Witten, I.H., Frank, E., Hall, M.A., Pal, C.J.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2016)Google Scholar
  14. 14.
    Xu, H., Xu, L.: Multi-label feature selection algorithm based on label pairwise ranking comparison transformation. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1210–1217. IEEE (2017)Google Scholar
  15. 15.
    Zaragoza, J.H., Sucar, L.E., Morales, E.F., Bielza, C., Larranaga, P.: Bayesian chain classifiers for multidimensional classification. IJCAI, vol. 11, pp. 2192–2197 (2011)Google Scholar
  16. 16.
    Zhang, M.L., Peña, J.M., Robles, V.: Feature selection for multi-label naive Bayes classification. Inf. Sci. 179(19), 3218–3229 (2009)CrossRefGoogle Scholar
  17. 17.
    Zhang, M.L., Zhou, Z.H.: Ml-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40(7), 2038–2048 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Ricardo Benítez Jiménez
    • 1
    Email author
  • Eduardo F. Morales
    • 1
  • Hugo Jair Escalante
    • 1
  1. 1.Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE)PueblaMexico

Personalised recommendations