Statistics and Computing

, Volume 17, Issue 3, pp 219–233 | Cite as

Inference in hybrid Bayesian networks using dynamic discretization

Article

Abstract

We consider approximate inference in hybrid Bayesian Networks (BNs) and present a new iterative algorithm that efficiently combines dynamic discretization with robust propagation algorithms on junction trees. Our approach offers a significant extension to Bayesian Network theory and practice by offering a flexible way of modeling continuous nodes in BNs conditioned on complex configurations of evidence and intermixed with discrete nodes as both parents and children of continuous nodes. Our algorithm is implemented in a commercial Bayesian Network software package, AgenaRisk, which allows model construction and testing to be carried out easily. The results from the empirical trials clearly show how our software can deal effectively with different type of hybrid models containing elements of expert judgment as well as statistical inference. In particular, we show how the rapid convergence of the algorithm towards zones of high probability density, make robust inference analysis possible even in situations where, due to the lack of information in both prior and data, robust sampling becomes unfeasible.

Keywords

Bayesian networks Expert systems Bayesian software Reasoning under uncertainty Statistical inference Propagation algorithms Dynamic discretization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agena Ltd. AgenaRisk software package. www.agenarisk.com (2005)
  2. Bernardo, J., Smith, A.: Bayesian Theory. Wiley, New York (1994) MATHGoogle Scholar
  3. Brigham, E.: Fast Fourier Transform and Its Applications, 1st edn. Prentice Hall, New York (1988) Google Scholar
  4. Buntine, W.: Operations for learning with graphical models. Artif. Intell. 2, 159–225 (1994) Google Scholar
  5. Casella, G., George, E.I.: Explaining the Gibbs sampler. Am. Stat. 46, 167–174 (1992) CrossRefGoogle Scholar
  6. Cobb, B., Shenoy, P.: Inference in hybrid Bayesian networks with mixtures of truncated exponentials. University of Kansas School of Business, working paper 294 (2005a) Google Scholar
  7. Cobb, B., Shenoy, P.: Nonlinear deterministic relationships in Bayesian networks. In: Godo, L. (Ed.) ECSQARU, pp. 27–38. Springer, Berlin (2005b) Google Scholar
  8. Dobson, A.J.: An Introduction to Generalized Linear Models. Chapman & Hall, New York (1990) MATHGoogle Scholar
  9. Fenton, N., Krause, P., Neil, M.: Probabilistic modeling for software quality control. J. Appl. Non-Classical Log. 12(2), 173–188 (2002) MATHCrossRefGoogle Scholar
  10. Fenton, N., Marsh, W., Neil, M., Cates, P., Forey, S., Tailor, T.: Making resource decisions for software projects. In: 26th International Conference on Software Engineering, Edinburgh, United Kingdom (2004) Google Scholar
  11. Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B.: Bayesian Data Analysis, 2nd edn., pp. 209–302. Chapman and Hall, New York (2004) MATHGoogle Scholar
  12. Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Markov Chain Monte Carlo in Practice. Chapman and Hall, London (1996) MATHGoogle Scholar
  13. Heckerman, D., Mamdani, A., Wellman, M.P.: Real-world applications’ of Bayesian networks. Commun. ACM 38(3), 24–68 (1995) CrossRefGoogle Scholar
  14. Huang, C., Darwiche, A.: Inference in belief networks: A procedural guide. Int. J. Approx. Reason. 15(3), 225–263 (1996) MATHCrossRefGoogle Scholar
  15. Hugin: www.hugin.com (2005) Google Scholar
  16. Jensen, F.: An Introduction to Bayesian Networks. Springer, Berlin (1996) Google Scholar
  17. Jensen, F., Lauritzen, S.L., Olesen, K.: Bayesian updating in recursive graphical models by local computations. Comput. Stat. Quart. 4, 260–282 (1990) Google Scholar
  18. Koller, D., Lerner, U., Angelov, D.: A general algorithm for approximate inference and its applications to hybrid Bayes nets. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp.  324–333 (1999) Google Scholar
  19. Kozlov, A.V., Koller, D.: Nonuniform dynamic discretization in hybrid networks. In: Geiger, D., Shenoy, P.P. (eds.) Uncertain. Artif. Intell. 13, 314–325 (1997) Google Scholar
  20. Lauritzen, S.L.: Graphical Models, Oxford (1996) Google Scholar
  21. Lauritzen, S.L., Jensen, F.: Stable local computation with conditional Gaussian distributions. Stat. Comput. 11, 191–203 (2001) CrossRefGoogle Scholar
  22. Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structures and their application to expert systems (with discussion). J. Roy. Stat. Soc. Ser. B 50(2), 157–224 (1988) MATHGoogle Scholar
  23. Moral, S., Rumı, R., Salmeron, A., Mixtures of truncated exponentials in hybrid Bayesian networks. In: Besnard, P., Benferhart, S. (eds.) Symbolic and Quantitative Approaches to Reasoning under Uncertainty. Lecture Notes in Artificial Intelligence, vol. 2143, pp. 156–167 (2001) Google Scholar
  24. Murphy, K.: A variational approximation for Bayesian networks with discrete and continuous latent variables. In: Laskey, K.B., Prade, H. (eds.) Uncertainty in Artificial Intelligence, vol. 15, pp. 467–475 (1999) Google Scholar
  25. Netica: www.norsys.com (2005)
  26. Neil, M., Fenton, N., Forey, S., Harris, R.: Using Bayesian belief networks to predict the reliability of military vehicles. IEE Comput. Control Eng. J. 12(1), 11–20 (2001) CrossRefGoogle Scholar
  27. Neil, M., Malcolm, B., Shaw, R.: Modelling an air traffic control environment using Bayesian belief networks. In: 21st International System Safety Conference, Ottawa, ON, Canada (2003a) Google Scholar
  28. Neil, M., Krause, P., Fenton, N.: Software quality prediction using Bayesian networks in software engineering with computational intelligence. In: (ed.) Khoshgoftaar, T.M. The Kluwer International Series in Engineering and Computer Science, vol. 73 (2003b) Google Scholar
  29. Pearl, J.: Fusion, propogation and structuring in belief networks. Artif. Intell. 29, 241–288 (1986) MATHCrossRefGoogle Scholar
  30. Pearl, J.: Graphical models, causality, and intervention. Stat. Sci. 8(3), 266–273 (1993) Google Scholar
  31. Spiegelhalter, D.J., Thomas, A., Best, N.G., Gilks, W.R.: BUGS: Bayesian inference using Gibbs sampling, Version 0.50. MRC Biostatistics Unit, Cambridge (1995) Google Scholar
  32. Spiegelhalter, D.J., Lauritzen, S.L.: Sequential updating of conditional probabilities on directed graphical structures. Networks 20, 579–605 (1990) MATHCrossRefGoogle Scholar
  33. Shacter, R., Peot, M.: 1989. Simulation approaches to general probabilistic inference on belief networks. In: Proceedings of the 5th Annual Conference on Uncertainty in AI (UAI), pp. 221–230 Google Scholar
  34. Shenoy, P.: Binary join trees for computing marginals in the Shenoy–Shafer architecture. Int. J. Approx. Reason. 17(1), 1–25 (1997) CrossRefGoogle Scholar
  35. Shenoy, P., Shafer, G.: Axioms for Probability and Belief-Function Propagation, Readings in Uncertain Reasoning, pp. 575–610. Kaufmann, Los Altos (1990) Google Scholar
  36. Whittaker, J.: Graphical Models in Applied Multivariate Statistics. Wiley, New York (1990) MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer ScienceQueen Mary, University of LondonLondonUK
  2. 2.Agena LimitedLondonUK

Personalised recommendations