Advertisement

Context-Specific and Local Independence in Markovian Dependence Structures

  • Henrik NymanEmail author
  • Johan Pensar
  • Jukka Corander
Chapter

Abstract

Directed acyclic graphs (DAGs) have been established as one of the primary tools for characterizing dependencies and causality among variables in multivariate systems. However, it has also been recognized that DAGs may hide more nuanced forms of independence that are important for interpretation and operational efficiency of the dependence models. Such independencies are typically context-specific, meaning that a variable may lose its connection to another variable in a particular context determined by some other set of variables. Here we review context-specific independence in different classes of Markovian probability models both for static and spatially or temporally organized variables, including Bayesian networks, Markov networks, and higher-order Markov chains. The generality of the context-specific independence as a concept may spawn new ways to characterize dependence systems also beyond these traditional models, for example, in dependence logic.

Keywords

Bayesian Network Leaf Node Marginal Likelihood Chordal Graph Conditional Probability Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

H.N. and J.P. were funded by the Foundation of Åbo Akademi University, as part of the grant for the Center of Excellence in Optimization and Systems Engineering. J.P. was also funded by the Magnus Ehrnrooth foundation. J.C was funded by ERC grant 239784 and the COIN centre of excellence (grant 251170 from Academy of Finland).

References

  1. 1.
    Bacallado, S.: Bayesian analysis of variable-order, reversible Markov chains. Ann. Stat. 39 (2), 838–864 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Proceedings of the Twelfth Annual Conference on Uncertainty in Artificial Intelligence, pp. 115–123. Morgan Kaufmann, San Francisco, CA (1996)Google Scholar
  3. 3.
    Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and regression trees. CRC, Wadsworth (1984)zbMATHGoogle Scholar
  4. 4.
    Bühlmann, P., Wyner, A.J.: Variable length Markov chains. Ann. Stat. 27 (2), 480–513 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 52–60. Morgan Kaufmann, San Francisco, CA (1991)Google Scholar
  6. 6.
    Chickering, D.M., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. In: Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, pp. 80–89. Morgan Kaufmann, San Francisco, CA (1997)Google Scholar
  7. 7.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9 (4), 309–347 (1992)zbMATHGoogle Scholar
  8. 8.
    Corander, J.: Bayesian graphical model determination using decision theory. J. Multivar. Anal. 85 (2), 253–266 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Corander, J.: Labelled graphical models. Scand. J. Stat. 30, 493–508 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Csiszár, I.: I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 (1), 146–158 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Didelez, V.: Graphical models for marked point processes based on local independence. J. R. Stat. Soc. Ser. B (Stat Methodol.) 70 (1), 245–264 (2008)Google Scholar
  12. 12.
    Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings of the Twelfth Annual Conference on Uncertainty in Artificial Intelligence, pp. 252–262. Morgan Kaufmann, San Francisco, CA (1996)Google Scholar
  13. 13.
    Golumbic, M.C.: Algorithmic Graph Theory and Perfect Graphs, 2nd edn. Elsevier, Amsterdam (2004)zbMATHGoogle Scholar
  14. 14.
    Grädel, E., Väänänen, J.: Dependence and independence. Stud. Logica 101 (2), 300–410 (2012)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20 (3), 197–243 (1995)zbMATHGoogle Scholar
  16. 16.
    Jääskinen, V., Xiong, J., Corander, J., Koski, T.: Sparse Markov chains for sequence data. Scand. J. Stat. 41, 639–655 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT, Cambridge (2009)zbMATHGoogle Scholar
  18. 18.
    Kontinen, J., Väänänen, J.: Axiomatizing first-order consequences in dependence logic. Ann. Pure Appl. Logic 164 (11), 1101–1117 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Koski, T., Noble, J.: Bayesian Networks: An Introduction. Wiley, Chippenham (2009)CrossRefzbMATHGoogle Scholar
  20. 20.
    Lauritzen, S.L.: Graphical Models. Oxford University Press, Oxford (1996)zbMATHGoogle Scholar
  21. 21.
    Nyman, H., Pensar, J., Koski, T., Corander, J.: Stratified graphical models - context-specific independence in graphical models. Bayesian Anal. 9 (4), 883–908 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Nyman, H., Pensar, J., Koski, T., Corander, J.: Context-specific independence in graphical log-linear models. Comput. Stat. (2015). doi: 10.1007/s00180-015-0606-6 zbMATHGoogle Scholar
  23. 23.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, CA (1988)zbMATHGoogle Scholar
  24. 24.
    Pensar, J., Nyman, H., Koski, T., Corander, J.: Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models. Data Min. Knowl. Disc. 29 (2), 503–533 (2015)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Poole, D.: Probabilistic partial evaluation: exploiting rule structure in probabilistic inference. In: Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 1284–1291 (1997)Google Scholar
  26. 26.
    Poole, D., Zhang, N.: Exploiting contextual independence in probabilistic inference. J. Artif. Intell. Res. 18, 263–313 (2003)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Quinlan, J.R., Rivest, R.L.: Inferring decision trees using the minimum description length principle. Inf. Comput. 80, 227–248 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Rissanen, J.: A universal data compression system. IEEE Trans. Inf. Theory 29 (5), 656–664 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Rudas, T.: A new algorithm for the maximum likelihood estimation of graphical log-linear models. Comput. Stat. 13, 529–537 (1998)zbMATHGoogle Scholar
  30. 30.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Väänänen, J.: Dependence Logic. Cambridge University Press, Cambridge (2007)CrossRefzbMATHGoogle Scholar
  32. 32.
    Weinberger, M.J., Rissanen, J., Feder, M.: A universal finite memory source. IEEE Trans. Inf. Theory 41 (3), 643–652 (1995)CrossRefzbMATHGoogle Scholar
  33. 33.
    Whittaker, J.: Graphical Models in Applied Multivariate Statistics. Wiley, Chichester (1990)zbMATHGoogle Scholar
  34. 34.
    Zhang, N., Poole, D.: On the role of context-specific independence in probabilistic inference. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pp. 1288–1293 (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Mathematics and StatisticsÅbo Akademi UniversityTurkuFinland
  2. 2.Department of Mathematics and StatisticsUniversity of HelsinkiHelsinkiFinland

Personalised recommendations