Skip to main content

Learning tractable NAT-modeled Bayesian networks

Abstract

Bayesian networks (BNs) encode conditional independence to avoid combinatorial explosion on the number of variables, but are subject to exponential growth of space and inference time on the number of causes per effect variable. Among space-efficient local models, we focus on the Non-Impeding Noisy-AND Tree (NIN-AND Tree or NAT) models, due to their multiple merits, and on NAT-modeled BNs, where each multi-parent variable family may be encoded as a NAT-model. Although BN inference is generally exponential on treewidth, the inference is tractable with NAT-modeled BNs of high treewidth and low density. In this work, we present the first study to learn NAT-modeled BNs from data. We apply the MDL principle to learning NAT-modeled BNs by developing a corresponding scoring function, and we couple it with heuristic structure search. We show that when data satisfy NAT causal independence, high treewidth, and low density structure, learning underlying NAT modeled BNs is feasible.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence, pp. 115–123 (1996)

  2. 2.

    Chickering, D.: A transformational characterization of equivalent Bayesian network structures. In: Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, pp. 87–98 (1995)

  3. 3.

    Chickering, D., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. In: Proceedings of 13th Conference on Uncertainty in Artificial Intelligence, pp. 80–89 (1997)

  4. 4.

    Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings 12th Conference on Uncertainty in Artificial Intelligence, pp. 252–262. Morgan Kaufmann (1996)

  5. 5.

    Friedman, N., Yakhini, Z.: On the sample complexity of learning Bayesian networks. In: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence, pp. 274–282. Morgan Kaufmann (1996)

  6. 6.

    Heckerman, D., Geiger, D., Chickering, D.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20, 197–243 (1995)

    MATH  Google Scholar 

  7. 7.

    Henrion, M.: Some practical issues in constructing belief networks. In: Kanal, L., Levitt, T., Lemmer, J. (eds.) Uncertainty in Artificial Intelligence 3, pp. 161–173, Elsevier Science Publishers (1989)

  8. 8.

    Lee, C., van Beek, P.: Metaheuristics for score-and-search Bayesian network structure learning. In: Proceedings 30th Canadian Conference on Artificial Intelligence, pp. 129–141 (2017)

  9. 9.

    Maaskant, P., Druzdzel, M.: An independence of causal interactions model for opposing influences. In: Jaeger, M., Nielsen, T. (eds.) Proceedings of the 4th European Workshop on Probabilistic Graphical Models, pp 185–192. Hirtshals, Denmark (2008)

  10. 10.

    Rissanen, J.: Modeling by shortest data description. Automatica 14, 465–471 (1978)

    Article  Google Scholar 

  11. 11.

    Vomlel, J., Tichavsky, P.: An approximate tensor-based inference method applied to the game of Minesweeper. In: Proceedings of the 7th European Workshop on Probabilistic Graphical Models, pp. 535–550. Springer LNAI 8745 (2012)

  12. 12.

    Woudenberg, S., van der Gaag, L., Rademaker, C.: An intercausal cancellation model for Bayesian-network engineering. Inter. J. Approx Reason 63, 32–47 (2015)

    MathSciNet  Article  Google Scholar 

  13. 13.

    Xiang, Y.: Non-impeding noisy-AND tree causal models over multi-valued variables. Int. J. Approx. Reason. 53(7), 988–1002 (2012)

    MathSciNet  Article  Google Scholar 

  14. 14.

    Xiang, Y.: Direct causal structure extraction from pairwise interaction patterns in NAT modeling Bayesian networks. Int. J. Approx. Reason. 105, 175–193 (2019)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Xiang, Y., Jiang, Q.: NAT model based compression of Bayesian network CPTs over multi-valued variables. Comput. Intell. 34(1), 219–240 (2018)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Xiang, Y., Loker, D.: De-causalizing NAT-modeled Bayesian networks for inference efficiency. In: Bagheri, E., Cheung, J. (eds.) Canadian AI 2018, LNAI 10832, pp. 17–30. Springer (2018)

Download references

Acknowledgements

Financial support from the NSERC Discovery Grant to the first author is acknowledged.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yang Xiang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xiang, Y., Wang, Q. Learning tractable NAT-modeled Bayesian networks. Ann Math Artif Intell (2021). https://doi.org/10.1007/s10472-021-09748-0

Download citation

Keywords

  • Bayesian networks
  • Causal independence models
  • Probabilistic inference
  • Local structures
  • Machine Learning

Mathematics Subject Classification (2010)

  • 68T05
  • 68T37