Abstract
Causal independence modelling is a well-known method for reducing the size of probability tables, simplifying the probabilistic inference and explaining the underlying mechanisms in Bayesian networks. Recently, a generalization of the widely-used noisy OR and noisy AND models, causal independence models based on symmetric Boolean functions, was proposed. In this paper, we study the problem of learning the parameters in these models, further referred to as symmetric causal independence models. We present a computationally efficient EM algorithm to learn parameters in symmetric causal independence models, where the computational scheme of the Poisson binomial distribution is used to compute the conditional probabilities in the E-step. We study computational complexity and convergence of the developed algorithm. The presented EM algorithm allows us to assess the practical usefulness of symmetric causal independence models. In the assessment, the models are applied to a classification task; they perform competitively with state-of-the-art classifiers.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. Journal of Mathematical Psychology, 12, 387–415.
Bast, R. C., Kufe, D. W., Pollock, R. E., Weichselbaum, R. R., Holland, J. F., & Frei, E. (2000). Cancer medicine—5 review. Ontario: B.C. Decker.
Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
Chen, X. H., Dempster, A. P., & Liu, J. S. (1994). Weighted finite population sampling to maximize entropy. Biometrika, 81, 457–469.
Darroch, J. (1964). On the distribution of the number of successes in independent trials. The Annals of Mathematical Statistics, 35, 1317–1321.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39, 1–38.
Díez, F. J. (1993). Parameter adjustment in Bayes networks. The generalized noisy OR-gate. In Proceedings of the ninth conference on uncertainty in artificial intelligence (pp. 99–105).
Edwards, A. W. P. (1960). The meaning of binomial distribution. Nature, 186, 1074.
Enderton, H. B. (1972). A mathematical introduction to logic. San Diego: Academic.
Heckerman, D. (1993). Causal independence for knowledge acquisition and inference. In Proceedings of the ninth conference on uncertainty in artificial intelligence (pp. 122–127).
Heckerman, D., & Breese, J. S. (1994). A new look at causal independence. In Proceedings of the tenth conference on uncertainty in artificial intelligence (pp. 286–292).
Henrion, M. (1989). Some practical issues in constructing belief networks. Uncertainty in Artificial Intelligence, 3, 161–173.
Howard, S. (1972). Discussion on Professor Cox’s paper. Journal of the Royal Statistical Society, Series B, 34, 210–211.
Jurgelenaite, R., Lucas, P. J. F., & Heskes, T. (2006). Noisy threshold functions for modelling causal independence in Bayesian networks (Technical report ICIS-R06014). Radboud University Nijmegen.
Kappen, H. J., & Neijt, J. P. (2002). Promedas, a probabilistic decision support system for medical diagnosis (Technical report). SNN—UMCU.
Karčiauskas, G. (2002). Text categorization using hierarchical Bayesian networks classifiers. Master thesis, Aalborg University.
Le Cam, L. (1960). An approximation theorem for the Poisson binomial distribution. Pacific Journal of Mathematics, 10, 1181–1197.
Lucas, P. J. F. (2005). Bayesian network modelling through qualitative patterns. Artificial Intelligence, 163, 233–263.
Lucas, P. J. F., Boot, H., & Taal, B. (1998). Computer-based decision support in management of primary gastric non-Hodgkin lymphoma. Methods of Information in Medicine, 37, 206–219.
Meek, C., & Heckerman, D. (1997). Structure and parameter learning for causal independence and causal interaction models. In Proceedings of the thirteenth conference on uncertainty in artificial intelligence (pp. 366–375).
Neal, R. M., & Hinton, G. E. (1998). A view of the EM algorithm that justifies incremental, sparse, and other variants. In M. I. Jordan (Ed.), Learning in graphical models. Cambridge: MIT Press.
Olesen, K. G., Kjærulff, U., Jensen, F., Falck, B., Andreassen, S., & Andersen, S. K. (1989). A Munin network for the median nerve—a case study on loops. Applied Artificial Intelligence, 3, 384–403.
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. San Mateo: Morgan Kaufman.
Salzberg, S. L. (1997). On comparing classifiers: pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery, 1, 317–327.
Shwe, M. A., Middleton, B., Heckerman, D. E., Henrion, M., Horvitz, E. J., Lehmann, H. P., & Cooper, G. F. (1991). Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. I—The probabilistic model and inference algorithms. Methods of Information in Medicine, 30, 241–255.
Srinivas, S. (1993). A generalization of the noisy-or model. In Proceedings of the ninth conference on uncertainty in artificial intelligence (pp. 208–215).
Wegener, I. (1987). The complexity of Boolean functions. New York: Wiley.
Witten, I. H., & Frank, E. (2005). Data mining: practical machine learning tools and techniques. San Francisco: Morgan Kaufmann.
Visscher, S., Lucas, P. J. F., Bonten, M., & Schurink, K. (2005). Improving the therapeutic performance of a medical Bayesian network using noisy threshold models. In Proceedings of ISBMDA 2005, the 6th international symposium on biological and medical data analysis (pp. 161–172).
Vomlel, J. (2006). Noisy-or classifier. International Journal of Intelligent Systems, 21, 381–398.
Zagorecki, A., Voortman, M., & Druzdzel, M. J. (2006). Decomposing local probability distributions in Bayesian networks for improved inference and parameter learning. In Proceedings of the nineteenth international Florida artificial intelligence research society conference (pp. 860–865).
Zhang, N. L., & Poole, D. (1996). Exploiting causal independence in Bayesian networks inference. Journal of Artificial Intelligence Research, 5, 301–328.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editor: Kevin Murphy
This research, carried out in the TimeBayes project, was supported by the Netherlands Organization for Scientific Research (NWO) under project number FN4556 and a Vici grant (639.023.604) awarded to the second author.
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Jurgelenaite, R., Heskes, T. Learning symmetric causal independence models. Mach Learn 71, 133–153 (2008). https://doi.org/10.1007/s10994-007-5041-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-007-5041-7