Abstract
This work proposes an extended version of the well-known tree-augmented naive Bayes (TAN) classifier where the structure learning step is performed without requiring features to be connected to the class. Based on a modification of Edmonds’ algorithm, our structure learning procedure explores a superset of the structures that are considered by TAN, yet achieves global optimality of the learning score function in a very efficient way (quadratic in the number of features, the same complexity as learning TANs). A range of experiments show that we obtain models with better accuracy than TAN and comparable to the accuracy of the state-of-the-art classifier averaged one-dependence estimator.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29(2/3), 103–130 (1997)
Hand, D., Yu, K.: Idiot’s Bayes-Not So Stupid After All? International Statistical Review 69(3), 385–398 (2001)
Friedman, J.: On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1, 55–77 (1997)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29(2), 131–163 (1997)
Corani, G., de Campos, C.P.: A tree augmented classifier based on extreme imprecise Dirichlet model. Int. J. Approx. Reasoning 51, 1053–1068 (2010)
Madden, M.: On the classification performance of TAN and general Bayesian networks. Knowledge-Based Systems 22(7), 489–495 (2009)
Corani, G., Antonucci, A., Maua, D., Gabaglio, S.: Trading off speed and accuracy in multilabel classification. In: Proceedings of the 7th European Workshop on Probabilistic Graphical Models (2014)
Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the 8th Conference on Uncertainty in Artificial Intelligence, UAI 1992, pp. 52–60. Morgan Kaufmann, San Francisco (1991)
Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Kontkanen, P., Myllymäki, P., Silander, T., Tirri, H.: On supervised selection of Bayesian networks. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 334–342. Morgan Kaufmann Publishers Inc. (1999)
Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 973–978. Morgan Kaufmann (2001)
Turney, P.D.: Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research 2, 369–409 (1995)
Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory IT-14(3), 462–467 (1968)
Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive Bayes: Aggregating one-dependence estimators. Machine Learning 58(1), 5–24 (2005)
de Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research 12, 663–689 (2011)
Lucas, P.J.F.: Restricted Bayesian network structure learning. In: Gámez, J.A., Moral, S., Salmerón, A. (eds.) Advances in Bayesian Networks. STUDFUZZ, vol. 146, pp. 217–232. Springer, Berlin (2004)
Edmonds, J.: Optimum branchings. Journal of Research of the National Bureau of Standards B 71B(4), 233–240 (1967)
Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Science Sinica 14, 1396–1400 (1965)
Zwick, U.: Lecture notes on Analysis of Algorithms: Directed Minimum Spanning Trees (April 22, 2013)
Tarjan, R.E.: Finding optimum branchings. Networks 7, 25–35 (1977)
Camerini, P.M., Fratta, L., Maffioli, F.: A note on finding optimum branchings. Networks 9, 309–312 (1979)
Gabow, H.N., Galil, Z., Spencer, T., Tarjan, R.E.: Efficient algorithms for finding minimum spanning trees in undirected and directed graphs. Combinatorica 6(2), 109–122 (1986)
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Dasgupta, S.: Learning polytrees. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 134–141. Morgan Kaufmann, San Francisco (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
de Campos, C.P., Cuccu, M., Corani, G., Zaffalon, M. (2014). Extended Tree Augmented Naive Classifier. In: van der Gaag, L.C., Feelders, A.J. (eds) Probabilistic Graphical Models. PGM 2014. Lecture Notes in Computer Science(), vol 8754. Springer, Cham. https://doi.org/10.1007/978-3-319-11433-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-11433-0_12
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11432-3
Online ISBN: 978-3-319-11433-0
eBook Packages: Computer ScienceComputer Science (R0)