Advertisement

Ancestor Relations in the Presence of Unobserved Variables

Conference paper
  • 2.5k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6912)

Abstract

Bayesian networks (BNs) are an appealing model for causal and non-causal dependencies among a set of variables. Learning BNs from observational data is challenging due to the nonidentifiability of the network structure and model misspecification in the presence of unobserved (latent) variables. Here, we investigate the prospects of Bayesian learning of ancestor relations, including arcs, in the presence and absence of unobserved variables. An exact dynamic programming algorithm to compute the respective posterior probabilities is developed, under the complete data assumption. Our experimental results show that ancestor relations between observed variables, arcs in particular, can be learned with good power even when a majority of the involved variables are unobserved. For comparison, deduction of ancestor relations from single maximum a posteriori network structures or their Markov equivalence class appears somewhat inferior to Bayesian averaging. We also discuss some shortcomings of applying existing conditional independence test based methods for learning ancestor relations.

References

  1. 1.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9(4), 309–347 (1992)zbMATHGoogle Scholar
  2. 2.
    Eaton, D., Murphy, K.: Bayesian structure learning using dynamic programming and MCMC. In: Proceedings of the Twenty-Third Conference Annual Conference on Uncertainty in Artificial Intelligence, UAI (2007)Google Scholar
  3. 3.
    Elidan, G., Friedman, N.: Learning the dimensionality of hidden variables. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI), pp. 144–151 (2001)Google Scholar
  4. 4.
    Elidan, G., Friedman, N.: Learning hidden variable networks: The information bottleneck approach. Journal of Machine Learning Research 6, 81–127 (2005)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Elidan, G., Lotner, N., Friedman, N., Koller, D.: Discovering hidden variables: A structure-based approach. In: Advances in Neural Information Processing Systems (NIPS 2000), vol. 13 (2000)Google Scholar
  6. 6.
    Ellis, B., Wong, W.H.: Learning causal Bayesian network structures from data. Journal of American Statistical Association 103, 778–789 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  8. 8.
    Friedman, N.: Learning belief networks in the presence of missing values and hidden variables. In: Proceedings of the Fourteenth International Conference on Machine Learning, ICML (1997)Google Scholar
  9. 9.
    Friedman, N., Koller, D.: Being Bayesian about network structure: A Bayesian approach to structure discovery in Bayesian networks. Machine Learning 50(1-2), 95–125 (2003)CrossRefzbMATHGoogle Scholar
  10. 10.
    Grzegorczyk, M., Husmeier, D.: Improving the structure MCMC sampler for Bayesian networks by int roducing a new edge reversal move. Machine Learning 71, 265–305 (2008)CrossRefGoogle Scholar
  11. 11.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20(3), 197–243 (1995)zbMATHGoogle Scholar
  12. 12.
    Kang, E.Y., Shpitser, I., Eskin, E.: Respecting Markov equivalence in computing posterior probabilities of causal graphical features. In: Proceeding of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), pp. 1175–1180 (2010)Google Scholar
  13. 13.
    Kennes, R.: Computational aspects of the Möbius transformation of graphs. IEEE Transaction on Systems, Man, and Cybernetics 22(2), 201–223 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Koivisto, M.: Advances in exact Bayesian structure discovery in Bayesian networks. In: Proceedings of the Twenty-Second Annual Conference on Uncertainty in Artificial Intelligence, UAI (2006)Google Scholar
  15. 15.
    Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. Journal of Machine Learning Research 5, 549–573 (2004)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Madigan, D., York, J.: Bayesian graphical models for discrete data. International Statistical Review 63, 215–232 (1995)CrossRefzbMATHGoogle Scholar
  17. 17.
    Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  18. 18.
    Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. Springer, Heidelberg (2000)zbMATHGoogle Scholar
  19. 19.
    Spirtes, P., Meek, C., Richardson, T.: Causal inference in the presence of latent variables and selection bias. In: Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence, UAI (1995)Google Scholar
  20. 20.
    Tian, J., He, R.: Computing posterior probabilities of structural features in Bayesian networks. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, UAI (2009)Google Scholar
  21. 21.
    Tian, J., He, R., Ram, L.: Bayesian model averaging using the k-best Bayesian network structures. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, UAI (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Helsinki Institute for Information Technology HIIT, Department of Computer ScienceUniversity of HelsinkiFinland

Personalised recommendations