Skip to main content
SpringerLink
Log in

Probabilistic decision graphs for optimization under uncertainty

  • Invited Survey
  • Published:
4OR Aims and scope Submit manuscript

Abstract

This paper provides a survey on probabilistic decision graphs for modeling and solving decision problems under uncertainty. We give an introduction to influence diagrams, which is a popular framework for representing and solving sequential decision problems with a single decision maker. As the methods for solving influence diagrams can scale rather badly in the length of the decision sequence, we present a couple of approaches for calculating approximate solutions. The modeling scope of the influence diagram is limited to so-called symmetric decision problems. This limitation has motivated the development of alternative representation languages, which enlarge the class of decision problems that can be modeled efficiently. We present some of these alternative frameworks and demonstrate their expressibility using several examples. Finally, we provide a list of software systems that implement the frameworks described in the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahlmann-Ohlsen KS, Jensen FV, Nielsen TD, Pedersen O, Vomlelova M (2009) A comparsion of two approaches for solving unconstrained influence diagrams. Int J Approx Reason 50: 153–173

    Article  Google Scholar 

  • Bhattacharjya D, Shachter RD (2007) Evaluating influence diagrams with decision circuits. In: Proceedings of the twentythird conference on Uncertainty in Artificial Intelligence (UAI). pp 9–16

  • Bhattacharjya D, Shachter RD (2008) Sensitivity analysis in decision circuits. In: Proceedings of the twentyfourth conference on Uncertainty in Artificial Intelligence (UAI). pp 34–42

  • Bielza C, Ríos-Insua S, Gómez M, del Pozo JAF (2000) Sensitivity analysis in ictneo. In: Lecture notes in statistics, Robust Bayesian analysis, vol 152. Springer, Berlin, pp 317–334

  • Cano A, Gómez M, Moral S (2006) A forward-backward Monte Carlo method for solving influence diagrams. Int J Approx Reason 42(1–2): 119–135

    Article  Google Scholar 

  • Charnes JM, Shenoy PP (2004) Multistage monte carlo method for solving influence diagrams using local computation. Manag Sci 50: 405–418

    Article  Google Scholar 

  • Cobb B (2006) Continuous decision MTE influence diagrams. In: Proceedings of the third European workshop on probabilistic graphical models. pp 67–74

  • Cobb BR, Shenoy PP (2004) Hybrid influence diagrams using mixtures of truncated exponentials. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI). pp 85–93

  • Cobb BR, Shenoy PP (2008) Decision making with hybrid influence diagrams using mixtures of truncated exponentials. Eur J Oper Res 186(1): 261–275

    Article  Google Scholar 

  • Cooper GF (1988) A method for using belief networks as influence diagrams. In: Proceedings of the fourth conference on uncertainty in artificial intelligence. pp 55–63

  • Cooper GF (1990) The computational complexity of probabilistic inference using Bayesian belief networks. Artif Intell 42(2–3): 393–405

    Article  Google Scholar 

  • Darwiche A (2003) A differential approach to inference in Bayesian networks. J ACM 50(3): 280–305

    Article  Google Scholar 

  • Darwiche A (2009) Modeling and reasoning with Bayesian networks. Cambridge University Press, Cambridge

    Google Scholar 

  • Dechter R (2000) An anytime approximation for optimizing policies under uncertainty. In: In workshop on decision theoretic planning, at the 5th international conference on artificial intelligence planning systems (AIPS-2000)

  • Dittmer SL, Jensen FV (1997) Myopic value of information in influence diagrams. In: Geiger D, Shenoy PP (eds) Proceedings of the thirteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 142–149

    Google Scholar 

  • Felli JC, Hazen GB (1999) Sensitivity analysis and the expected value of perfect information. Med Decis Mak 18: 95–109

    Article  Google Scholar 

  • Gal Y, Pfeffer A (2003) A language for modeling agents’ decision making processes in games. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems. ACM Press, New york, pp 265 – 272

  • Gal Y, Pfeffer A (2008) Networks of influence diagrams: a formalism for representing agent’s beliefs and decision making processes. J Artif Intell Res 33: 109–147

    Google Scholar 

  • Garcia-Sanchez D, Druzdzel M (2007) An efficient exhaustive anytime sampling algorithm for influence diagrams. In: Lucas P, Gmez J, Salmern A (eds) Advances in probabilistic graphical models, studies in fuzziness and soft computing, vol 214. Springer, Berlin, pp 255–273

    Google Scholar 

  • Gmytrasiewicz PJ, Durfee EH, Wehe DK (1991) A decision-theoretic approach to coordinating multiagent interactions. In: Proceedings of the twelfth international joint conference on artificial intelligence. pp 62–68

  • Horsch MC, Poole D (1998) An anytime algorithm for decision making under uncertainty. In: Cooper GF, Moral S (eds) Proceedings of the fourteenth conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 246–255

    Google Scholar 

  • Howard RA, Matheson JE (1981) Influence diagrams. In: Howard RA, Matheson JE (eds) The principles and applications of decision analysis vol 2, chap 37. Strategic Decision Group, Menlo Park, pp 721–762

    Google Scholar 

  • Jensen F, Jensen FV, Dittmer SL (1994) From influence diagrams to junction trees. In: Mantaras RL, Poole D (eds) Proceedings of the tenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 367–373

    Google Scholar 

  • Jensen FV, Gatti E (2010) Information enhancement for approximate representation of optimal strategies for influence diagrams. In: Proceedings of the fifth European workshop on probabilistic graphical models. pp 102–9

  • Jensen FV, Nielsen TD (2007) Bayesian networks and decision graphs, 2nd edn. Springer, New York ISBN: 0-387-68281-3

    Book  Google Scholar 

  • Jensen FV, Vomlelova M (2002) Unconstrained influence diagrams. In: Darwiche A, Friedman N (eds) Proceedings of the eighteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 234–241

    Google Scholar 

  • Jensen FV, Nielsen TD, Shenoy PP (2006) Sequential influence diagrams: a unified framework. Int J Approx Reason 42: 101–118

    Article  Google Scholar 

  • Kjaerulff UB, Madsen AL (2008) Bayesian networks and influence diagrams, 1st edn. Springer, New York ISBN: 978-0-387-74100-0

    Book  Google Scholar 

  • Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge

    Google Scholar 

  • Koller D, Milch B (2001) Multi-agent influence diagrams for representing and solving games. In: Proceedings of the seventeenth international joint conference on artificial intelligence. pp 1027–1036

  • Koller D, Milch B (2003) Multi-agent influence diagrams for representing and solving games. Games Econ Behav 45(1): 181–221

    Article  Google Scholar 

  • Korb KB, Nicholson AE (2004) Bayesian artificial intelligence, 1st edn. Chapman and Hall, London ISBN:1-58488-387-1

    Google Scholar 

  • Lauritzen SL, Nilsson D (2001) Representing and solving decision problems with limited information. Manag Sci 47(9): 1235–1251

    Article  Google Scholar 

  • Li Y, Shenoy PP (2010) Solving hybrid influence diagrams with deterministic variables. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). pp 322–331

  • Liao W, Ji Q (2008) Efficient non-myopic value-of-information computation for influence diagrams. Int J Approx Reason 49: 436–450

    Article  Google Scholar 

  • Luque M, Díez FJ (2010) Variable elimination for influence diagrams with super value nodes. Int J Approx Reason 51: 615–631

    Article  Google Scholar 

  • Madsen AL (2008) New methods for marginalization in lazy propagation. In: Jaeger M, Nielsen TD (eds) Proceedings of the fourth European workshop on probabilistic graphical models. pp 193–200

  • Madsen AL, Jensen F (2005) Solving linear-quadratic conditional Gaussian influence diagrams. Int J Approx Reason 38: 263–282

    Article  Google Scholar 

  • Madsen AL, Jensen FV (1999) Lazy evaluation of symmetric Bayesian decision problems. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 382–390

    Google Scholar 

  • Madsen AL, Nilsson D (2001) Solving influence diagrams using HUGIN, Shafer-Shenoy, and lazy propagation. In: Proceedings of the seventeenth conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 337–345

  • Moral S, Rumí R, Salmerón A (2001) Mixtures of truncated exponentials in hybrid Bayesian networks. In: Proceedings of the sixth European conference on symbolic and quantitative approaches to reasoning with uncertainty, lecture notes in artificial intelligence, vol 2143. Springer, Berlin, pp 145–167

  • Nielsen TD (2001) Decomposition of influence diagrams. In: Benferhat S, Besnard P (eds) Proceedings of the sixth European conference on symbolic and quantitative approaches to reasoning with uncertainty. Springer, no. 2143 in Lecture Notes in Artificial Intelligence, pp 144–155

  • Nielsen TD, Jensen FV (1999) Well-defined decision scenarios. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth Conference on Uncertainty in Artificial Intelligence (UAI). Morgan Kaufmann Publishers, Los Altos, pp 502–511

  • Nielsen TD, Jensen FV (2003) Sensitivity analysis in influence diagrams. IEEE Trans Syst Man Cybern Part A Syst Hum 33(2): 223–234

    Article  Google Scholar 

  • Nilsson D, Lauritzen SL (2000) Evaluating influence diagrams using LIMIDs. In: Boutilier C, Goldszmidt M (eds) Proceedings of the sixteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 436–445

    Google Scholar 

  • Pearl J (1988) Probabilistic reasoning in intelligent systems. representation and reasoning. Morgan Kaufmann Publishers, San Mateo. ISBN 0-934613-73-7

  • Raiffa H, Schlaifer R (1961) Applied statistical decision theory. MIT press, Cambridge

    Google Scholar 

  • Shachter RD (1986) Evaluating influence diagrams. Oper Res 34(6): 871–882

    Article  Google Scholar 

  • Shachter RD (1999) Efficient value of information computation. In: Laskey KB, Prade H (eds) Proceedings of the fifteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 594–601

    Google Scholar 

  • Shachter RD, Bhattacharjya D (2010) Dynamic programming in influence diagrams with decision circuits. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). pp 509–516

  • Shachter RD, Kenley CR (1989) Gaussian influence diagrams. Manag Sci 35(5): 527–550

    Article  Google Scholar 

  • Shachter RD, Peot MA (1992) Decision making using probabilistic inference methods. In: Dubois D, Wellman MP, D’Ambrosio B, Smets P (eds) Proceedings of the eighth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 276–283

    Google Scholar 

  • Shafer GR, Shenoy PP (1990) Probability Propagation. Ann Math Artif Intell 2: 327–352

    Article  Google Scholar 

  • Shenoy PP (1992) Valuation-based systems for Bayesian decision analysis. Oper Res 40(3): 463–484

    Article  Google Scholar 

  • Shenoy PP (2000) Valuation network representation and solution of asymmetric decision problems. Eur J Oper Res 121(3): 579–608

    Article  Google Scholar 

  • Shenoy PP, West JC (2009) Mixtures of polynomials in hybrid bayesian networks with deterministic variables. In: Proceedings of the eighth workshop on uncertainty processing. pp 202–212

  • Virto MA, Martín J, Insua DR, Moreno-Díaz A (2002) Approximate solutions of complex influence diagrams through mcmc methods. In: Proceedings of the first European workshop on probabilistic graphical models

  • Von Neumann J, Morgenstern O (1944) Theory of games and economic behavior, 1st edn. Wiley, New York

    Google Scholar 

  • Yuan C, Wu X (2010) Solving influence diagrams using heuristic search. In: Proceedings of the eleventh international symposium on artificial intelligence and mathematics

  • Yuan C, Wu X, Hansen E (2010) Solving multistage influence diagrams using branch-and-bound search. In: Proceedings of the twentysixth conference on Uncertainty in Artificial Intelligence (UAI). Catalina Island, CA

  • Zhang NL (1998) Probabilistic inference in influence diagrams. In: Cooper GF, Moral S (eds) Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Los Altos, pp 514–522

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Aalborg University, Aalborg, Denmark

    Finn V. Jensen & Thomas Dyhre Nielsen

Authors
  1. Finn V. Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Dyhre Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Dyhre Nielsen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jensen, F.V., Nielsen, T.D. Probabilistic decision graphs for optimization under uncertainty. 4OR-Q J Oper Res 9, 1–28 (2011). https://doi.org/10.1007/s10288-011-0159-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10288-011-0159-7

Keywords

MSC classification (2000)

Search

Navigation