Abstract
In this paper, we present cutset networks, a new tractable probabilistic model for representing multi-dimensional discrete distributions. Cutset networks are rooted OR search trees, in which each OR node represents conditioning of a variable in the model, with tree Bayesian networks (Chow-Liu trees) at the leaves. From an inference point of view, cutset networks model the mechanics of Pearl’s cutset conditioning algorithm, a popular exact inference method for probabilistic graphical models. We present efficient algorithms, which leverage and adopt vast amount of research on decision tree induction for learning cutset networks from data. We also present an expectation-maximization (EM) algorithm for learning mixtures of cutset networks. Our experiments on a wide variety of benchmark datasets clearly demonstrate that compared to approaches for learning other tractable models such as thin-junction trees, latent tree models, arithmetic circuits and sum-product networks, our approach is significantly more scalable, and provides similar or better accuracy.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bach, F., Jordan, M.: Thin junction trees. Advances in Neural Information Processing Systems 14, 569–576 (2001)
Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pp. 115–123. Morgan Kaufmann, Portland (1996)
Chavira, M., Darwiche, A.: On probabilistic inference by weighted model counting. Artificial Intelligence 172(6-7), 772–799 (2008)
Chechetka, A., Guestrin, C.: Efficient principled learning of thin junction trees. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems 20. MIT Press, Cambridge (2008)
Choi, M.J., Tan, V., Anandkumar, A., Willsky, A.: Learning latent tree graphical models. Journal of Machine Learning Research 12, 1771–1812 (2011)
Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory 14, 462–467 (1968)
Darwiche, A.: Modeling and reasoning with Bayesian networks. Cambridge University Press (2009)
Davis, J., Domingos, P.: Bottom-up learning of Markov network structure. In: Proceedings of the Twenty-Seventh International Conference on Machine Learning, pp. 271–278. ACM Press, Haifa (2010)
Dechter, R., Mateescu, R.: AND/OR search spaces for graphical models. Artificial Intelligence 171(2), 73–106 (2007)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39, 1–38 (1977)
Gens, R., Domingos, P.: Learning the structure of sum-product networks. In: Proceedings of the Thirtieth International Conference on Machine Learning. JMLR: W&CP, vol. 28 (2013)
Gogate, V., Domingos, P.: Formula-Based Probabilistic Inference. In: Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, pp. 210–219 (2010)
Gogate, V., Webb, W., Domingos, P.: Learning efficient Markov networks. In: Proceedings of the 24th Conference on Neural Information Processing Systems, NIPS 2010 (2010)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)
Lowd, D., Davis, J.: Learning Markov network structure with decision trees. In: Proceedings of the 10th IEEE International Conference on Data Mining (ICDM), pp. 334–343. IEEE Computer Society Press, Sydney (2010)
Lowd, D., Rooshenas, A.: Learning Markov networks with arithmetic circuits. In: Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2013), Scottsdale, AZ (2013)
Meila, M., Jordan, M.: Learning with mixtures of trees. Journal of Machine Learning Research 1, 1–48 (2000)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Narasimhan, M., Bilmes, J.: Pac-learning bounded tree-width graphical models. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (2004)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)
Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)
Rooshenas, A., Lowd, D.: Learning sum-product networks with direct and indirect interactions. In: Proceedings of the Thirty-First International Conference on Machine Learning. ACM Press, Beijing (2014)
Van Haaren, J., Davis, J.: Markov network structure learning: A randomized feature generation approach. In: Proceedings of the Twenty-Sixth National Conference on Artificial Intelligence. AAAI Press (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rahman, T., Kothalkar, P., Gogate, V. (2014). Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44851-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-662-44851-9_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44850-2
Online ISBN: 978-3-662-44851-9
eBook Packages: Computer ScienceComputer Science (R0)