Extended Tree Augmented Naive Classifier

  • Cassio P. de Campos
  • Marco Cuccu
  • Giorgio Corani
  • Marco Zaffalon
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8754)


This work proposes an extended version of the well-known tree-augmented naive Bayes (TAN) classifier where the structure learning step is performed without requiring features to be connected to the class. Based on a modification of Edmonds’ algorithm, our structure learning procedure explores a superset of the structures that are considered by TAN, yet achieves global optimality of the learning score function in a very efficient way (quadratic in the number of features, the same complexity as learning TANs). A range of experiments show that we obtain models with better accuracy than TAN and comparable to the accuracy of the state-of-the-art classifier averaged one-dependence estimator.


Bayesian Network Score Function Directed Acyclic Graph Minimum Span Tree Minimum Span Tree Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29(2/3), 103–130 (1997)CrossRefzbMATHGoogle Scholar
  2. 2.
    Hand, D., Yu, K.: Idiot’s Bayes-Not So Stupid After All? International Statistical Review 69(3), 385–398 (2001)zbMATHGoogle Scholar
  3. 3.
    Friedman, J.: On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1, 55–77 (1997)CrossRefGoogle Scholar
  4. 4.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29(2), 131–163 (1997)CrossRefzbMATHGoogle Scholar
  5. 5.
    Corani, G., de Campos, C.P.: A tree augmented classifier based on extreme imprecise Dirichlet model. Int. J. Approx. Reasoning 51, 1053–1068 (2010)CrossRefzbMATHGoogle Scholar
  6. 6.
    Madden, M.: On the classification performance of TAN and general Bayesian networks. Knowledge-Based Systems 22(7), 489–495 (2009)CrossRefGoogle Scholar
  7. 7.
    Corani, G., Antonucci, A., Maua, D., Gabaglio, S.: Trading off speed and accuracy in multilabel classification. In: Proceedings of the 7th European Workshop on Probabilistic Graphical Models (2014)Google Scholar
  8. 8.
    Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the 8th Conference on Uncertainty in Artificial Intelligence, UAI 1992, pp. 52–60. Morgan Kaufmann, San Francisco (1991)Google Scholar
  9. 9.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)zbMATHGoogle Scholar
  10. 10.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)zbMATHGoogle Scholar
  11. 11.
    Kontkanen, P., Myllymäki, P., Silander, T., Tirri, H.: On supervised selection of Bayesian networks. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 334–342. Morgan Kaufmann Publishers Inc. (1999)Google Scholar
  12. 12.
    Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 973–978. Morgan Kaufmann (2001)Google Scholar
  13. 13.
    Turney, P.D.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)Google Scholar
  14. 14.
    Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory IT-14(3), 462–467 (1968)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive Bayes: Aggregating one-dependence estimators. Machine Learning 58(1), 5–24 (2005)CrossRefzbMATHGoogle Scholar
  16. 16.
    de Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research 12, 663–689 (2011)zbMATHGoogle Scholar
  17. 17.
    Lucas, P.J.F.: Restricted Bayesian network structure learning. In: Gámez, J.A., Moral, S., Salmerón, A. (eds.) Advances in Bayesian Networks. STUDFUZZ, vol. 146, pp. 217–232. Springer, Berlin (2004)CrossRefGoogle Scholar
  18. 18.
    Edmonds, J.: Optimum branchings. Journal of Research of the National Bureau of Standards B 71B(4), 233–240 (1967)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Science Sinica 14, 1396–1400 (1965)zbMATHGoogle Scholar
  20. 20.
    Zwick, U.: Lecture notes on Analysis of Algorithms: Directed Minimum Spanning Trees (April 22, 2013)Google Scholar
  21. 21.
    Tarjan, R.E.: Finding optimum branchings. Networks 7, 25–35 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Camerini, P.M., Fratta, L., Maffioli, F.: A note on finding optimum branchings. Networks 9, 309–312 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Gabow, H.N., Galil, Z., Spencer, T., Tarjan, R.E.: Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica 6(2), 109–122 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007),
  25. 25.
    Dasgupta, S.: Learning polytrees. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 134–141. Morgan Kaufmann, San Francisco (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Cassio P. de Campos
    • 1
  • Marco Cuccu
    • 2
  • Giorgio Corani
    • 1
  • Marco Zaffalon
    • 1
  1. 1.Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA)Switzerland
  2. 2.Università della Svizzera italiana (USI)Switzerland

Personalised recommendations