A Double Weighted Naive Bayes for Multi-label Classification

  • Xuesong Yan
  • Wei Li
  • Qinghua Wu
  • Victor S. Sheng
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 575)


Multi-label classification is to assign an instance to multiple classes. Naive Bayes (NB) is one of the most popular algorithms for pattern recognition and classification. It has a high performance in single label classification. It is naturally extended for multi-label classification under the assumption of label independence. As we know, NB is based on a simple but unrealistic assumption that attributes are conditionally independent given the class. Therefore, a double weighted NB (DWNB) is proposed to demonstrate the influences of predicting different labels based on different attributes. Our DWNB utilizes the niching cultural algorithm to determine the weight configuration automatically. Our experimental results show that our proposed DWNB outperforms NB and its extensions significantly in multi-label classification.


Multi-label classification Naive Bayes Cultural algorithm Double weighted Naive Bayes 



This paper is supported by Natural Science Foundation of China (No. 61402425, 61272470, 61305087, 61440060, 41404076), the Provincial Natural Science Foundation of Hubei (No. 2013CFB320, 2015CFA065).


  1. 1.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Reasoning. Morgan Kaufmann Publishers, Los Altos (1988)Google Scholar
  2. 2.
    Langley, P., Iba, W., Thompson, K.: An analysis of Bayesian classifiers. AAAI 90, 223–228 (1992)Google Scholar
  3. 3.
    Xie, Z., Hsu, W., Liu, Z., Li Lee, M.: SNNB: a selective neighborhood based Naive Bayes for lazy learning. In: Chen, M.-S., Yu, P.S., Liu, B. (eds.) PAKDD 2002. LNCS (LNAI), vol. 2336, pp. 104–114. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Langley, P., Sage, S.: Induction of selective Bayesian classifiers. In: Proceedings of the 10th International Conference on Uncertainty in Artificial Intelligence, pp. 399–406. Morgan Kaufmann Publishers Inc (1994)Google Scholar
  5. 5.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)CrossRefzbMATHGoogle Scholar
  6. 6.
    Kohavi, R.: Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid. In: KDD, pp. 202–207 (1996)Google Scholar
  7. 7.
    Sucar, L.E., Bielza, C., Morales, E.F., et al.: Multi-label classification with Bayesian network-based chain classifiers. Pattern Recogn. Lett. 41, 14–22 (2014)CrossRefGoogle Scholar
  8. 8.
    Qu, G., Zhang, H., Hartrick, C.T.: Multi-label classification with Bayesian theorem. In: 4th International Conference on Biomedical Engineering and Informatics (BMEI), vol. 4, pp. 2281–2285. IEEE (2011)Google Scholar
  9. 9.
    Zhang, H., Sheng, S.: Learning weighted Naive Bayes with accurate ranking. In: 4th IEEE International Conference on Data Mining, pp. 567–570. IEEE (2004)Google Scholar
  10. 10.
    Hall, M.: Correlation-based feature selection for discrete and numeric class machine‎ learning. In: ‎Proceedings‎ of‎ the 7th‎ Intentional‎ Conference‎ on‎ Machine‎ Learning,‎ Stanford University (2000)Google Scholar
  11. 11.
    Jiang, L., Zhang, H.: Weightily averaged one-dependence estimators. In: Yang, Q., Webb, G. (eds.) PRICAI 2006. LNCS (LNAI), vol. 4099, pp. 970–974. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53(1–2), 23–69 (2003)CrossRefzbMATHGoogle Scholar
  13. 13.
    Hall, M.: A decision tree-based attribute weighting filter for Naive Bayes. Knowl.-Based Syst. 20(2), 120–126 (2007)CrossRefGoogle Scholar
  14. 14.
    Wu, J., Cai, Z.: Attribute weighting via differential evolution algorithm for attribute weighted Naive Bayes (wnb). J. Comput. Inf. Syst. 7(5), 1672–1679 (2011)Google Scholar
  15. 15.
    Wu, J., Pan, S., Cai, Z., et al.: Dual instance and attribute weighting for Naive Bayes classification. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 1675–1679. IEEE (2014)Google Scholar
  16. 16.
    Reynoids, R.: An introduction to cultural algorithms. In: Sebald, A.X., Fogel, L.J. (eds.) Proceedings of the 3rd Annual Conference on Evolutionary Programming, pp. 13 1–139. World Scientific Publishing, River Edge, NJ (1994)Google Scholar
  17. 17.
    Chung, C.: Knowledge-based approaches to self-adaptation in cultural algorithms. Ph.D thesis, Wayne State University, Detroit, Michigan, USA (1997)Google Scholar
  18. 18.
    Zhang, Y.: Cultural algorithm and its application in the portfolio. Master thesis, Harbin University of Science and Technology, Harbin, China (2008)Google Scholar
  19. 19.
    Horn, J., Nafpliotis, N., Goldberg, D.E.: A niched Pareto genetic algorithm for multi-objective optimization. In: Proceedings of the IEEE World Congress on Computational Intelligence, pp. 82–87 (1994)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2016

Authors and Affiliations

  • Xuesong Yan
    • 1
  • Wei Li
    • 1
  • Qinghua Wu
    • 2
  • Victor S. Sheng
    • 3
  1. 1.School of Computer ScienceChin University of GeoscienceHubeiChina
  2. 2.Faculty of Computer Science and EngineeringWuHan Institute of TechnologyHubeiChina
  3. 3.Department of Computer ScienceUniversity of Central ArkansasConwayUSA

Personalised recommendations