Extended Tree Augmented Naive Classifier

  • Cassio P. de Campos
  • Marco Cuccu
  • Giorgio Corani
  • Marco Zaffalon
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8754)

Abstract

This work proposes an extended version of the well-known tree-augmented naive Bayes (TAN) classifier where the structure learning step is performed without requiring features to be connected to the class. Based on a modification of Edmonds’ algorithm, our structure learning procedure explores a superset of the structures that are considered by TAN, yet achieves global optimality of the learning score function in a very efficient way (quadratic in the number of features, the same complexity as learning TANs). A range of experiments show that we obtain models with better accuracy than TAN and comparable to the accuracy of the state-of-the-art classifier averaged one-dependence estimator.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29(2/3), 103–130 (1997)CrossRefMATHGoogle Scholar
  2. 2.
    Hand, D., Yu, K.: Idiot’s Bayes-Not So Stupid After All? International Statistical Review 69(3), 385–398 (2001)MATHGoogle Scholar
  3. 3.
    Friedman, J.: On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1, 55–77 (1997)CrossRefGoogle Scholar
  4. 4.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29(2), 131–163 (1997)CrossRefMATHGoogle Scholar
  5. 5.
    Corani, G., de Campos, C.P.: A tree augmented classifier based on extreme imprecise Dirichlet model. Int. J. Approx. Reasoning 51, 1053–1068 (2010)CrossRefMATHGoogle Scholar
  6. 6.
    Madden, M.: On the classification performance of TAN and general Bayesian networks. Knowledge-Based Systems 22(7), 489–495 (2009)CrossRefGoogle Scholar
  7. 7.
    Corani, G., Antonucci, A., Maua, D., Gabaglio, S.: Trading off speed and accuracy in multilabel classification. In: Proceedings of the 7th European Workshop on Probabilistic Graphical Models (2014)Google Scholar
  8. 8.
    Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the 8th Conference on Uncertainty in Artificial Intelligence, UAI 1992, pp. 52–60. Morgan Kaufmann, San Francisco (1991)Google Scholar
  9. 9.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)MATHGoogle Scholar
  10. 10.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)MATHGoogle Scholar
  11. 11.
    Kontkanen, P., Myllymäki, P., Silander, T., Tirri, H.: On supervised selection of Bayesian networks. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 334–342. Morgan Kaufmann Publishers Inc. (1999)Google Scholar
  12. 12.
    Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 973–978. Morgan Kaufmann (2001)Google Scholar
  13. 13.
    Turney, P.D.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)Google Scholar
  14. 14.
    Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory IT-14(3), 462–467 (1968)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive Bayes: Aggregating one-dependence estimators. Machine Learning 58(1), 5–24 (2005)CrossRefMATHGoogle Scholar
  16. 16.
    de Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research 12, 663–689 (2011)MATHGoogle Scholar
  17. 17.
    Lucas, P.J.F.: Restricted Bayesian network structure learning. In: Gámez, J.A., Moral, S., Salmerón, A. (eds.) Advances in Bayesian Networks. STUDFUZZ, vol. 146, pp. 217–232. Springer, Berlin (2004)CrossRefGoogle Scholar
  18. 18.
    Edmonds, J.: Optimum branchings. Journal of Research of the National Bureau of Standards B 71B(4), 233–240 (1967)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Science Sinica 14, 1396–1400 (1965)MATHGoogle Scholar
  20. 20.
    Zwick, U.: Lecture notes on Analysis of Algorithms: Directed Minimum Spanning Trees (April 22, 2013)Google Scholar
  21. 21.
    Tarjan, R.E.: Finding optimum branchings. Networks 7, 25–35 (1977)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Camerini, P.M., Fratta, L., Maffioli, F.: A note on finding optimum branchings. Networks 9, 309–312 (1979)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Gabow, H.N., Galil, Z., Spencer, T., Tarjan, R.E.: Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica 6(2), 109–122 (1986)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
  25. 25.
    Dasgupta, S.: Learning polytrees. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 134–141. Morgan Kaufmann, San Francisco (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Cassio P. de Campos
    • 1
  • Marco Cuccu
    • 2
  • Giorgio Corani
    • 1
  • Marco Zaffalon
    • 1
  1. 1.Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA)Switzerland
  2. 2.Università della Svizzera italiana (USI)Switzerland

Personalised recommendations