Unifying Nondeterministic and Probabilistic Planning Through Imprecise Markov Decision Processes

  • Felipe W. Trevizan
  • Fábio G. Cozman
  • Leliane N. de Barros
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4140)


This paper proposes an unifying formulation for nondeterministic and probabilistic planning. These two strands of AI planning have followed different strategies: while nondeterministic planning usually looks for minimax (or worst-case) policies, probabilistic planning attempts to maximize expected reward. In this paper we show that both problems are special cases of a more general approach, and we demonstrate that the resulting structures are Markov Decision Processes with Imprecise Probabilities (MDPIPs). We also show how existing algorithms for MDPIPs can be adapted to planning under uncertainty.


Partial Observability Probabilistic Planning Policy Improvement Imprecise Probability Nondeterministic Choice 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ghallab, M., Nau, D., Traverso, P.: Automated Planning: Theory & Practice. Morgan Kaufmann, San Francisco (2004)MATHGoogle Scholar
  2. 2.
    Bonet, B., Geffner, H.: Learning Depth-First Search: A unified approach to heuristic search in deterministic and non-deterministic settings, and its application to MDPs. In: ICAPS (2006) (to appear)Google Scholar
  3. 3.
    Bonet, B., Geffner, H.: Labeled RTDP: Improving the convergence of real-time dynamic programming. In: ICAPS, Trento, Italy, pp. 12–21. AAAI Press, Menlo Park (2003)Google Scholar
  4. 4.
    Guestrin, C., Koller, D., Parr, R., Venkataraman, S.: Efficient solution algorithms for factored MDPs. J. Artif. Intell. Res (JAIR) 19, 399–468 (2003)MATHMathSciNetGoogle Scholar
  5. 5.
    Bertoli, P., Cimatti, A., Roveri, M., Traverso, P.: Planning in nondeterministic domains under partial observability via symbolic model checking. In: IJCAI, pp. 473–478 (2001)Google Scholar
  6. 6.
    Bonet, B., Geffner, H.: Planning with incomplete information as heuristic search in belief space. In: ICAPS, Breckenridge, CO, pp. 52–61. AAAI Press, Menlo Park (2000)Google Scholar
  7. 7.
    Luce, D., Raiffa, H.: Games and Decisions. Dover edn., Mineola (1957)Google Scholar
  8. 8.
    Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer, Heidelberg (1985)MATHGoogle Scholar
  9. 9.
    Knight, F.H.: Risk, Uncertainty, and Profit. Hart, Schaffner & Marx. Houghton Mifflin Company, Boston (1921)Google Scholar
  10. 10.
    Levi, I.: The Enterprise of Knowledge. MIT Press, Cambridge (1980)Google Scholar
  11. 11.
    Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, London (1991)MATHGoogle Scholar
  12. 12.
    Walley, P.: Measures of uncertainty in expert systems. AI 83, 1–58 (1996)MathSciNetGoogle Scholar
  13. 13.
    Seidenfeld, T., Kadane, J.B., Schervish, M.J.: On the shared preferences of two Bayesian decision makers. The Journal of Philosophy 86(5), 225–244 (1989)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Seidenfeld, T., Schervish, M.: Two perspectives on consensus for (Bayesian) inference and decisions. IEEE Transactions on Systems, Man and Cybernetics 20(1), 318–325 (1990)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Huber, P.J.: Robust Statistics. Wiley, New York (1980)Google Scholar
  16. 16.
    Kadane, J.B. (ed.): Robustness of Bayesian Analyses. Studies in Bayesian econometrics, vol. 4. Elsevier Science Pub. Co., New York (1984)MATHGoogle Scholar
  17. 17.
    Frisch, A.M., Haddawy, P.: Anytime deduction for probabilistic logic. Artificial Intelligence 69, 93–122 (1994)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Halpern, J.Y.: Reasoning about uncertainty. MIT Press, Cambridge (2003)MATHGoogle Scholar
  19. 19.
    Nilsson, N.J.: Probabilistic logic. Artificial Intelligence 28, 71–87 (1986)MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)MATHGoogle Scholar
  21. 21.
    Anrig, B., Bissig, R., Haenni, R., Kohlas, J., Lehmann, N.: Probabilistic argumentation systems: Introduction to assumption-based modeling with ABEL. Technical Report 99-1, Institute of Informatics, University of Fribourg (1999)Google Scholar
  22. 22.
    Cozman, F.G.: Credal networks. AI 120, 199–233 (2000)MATHMathSciNetGoogle Scholar
  23. 23.
    Cozman, F.G.: Graphical models for imprecise probabilities. International Journal of Approximate Reasoning 39(2-3), 167–184 (2005)MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Fagiuoli, E., Zaffalon, M.: 2U: An exact interval propagation algorithm for polytrees with binary variables. Artificial Intelligence 106(1), 77–107 (1998)MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    de Cooman, G., Cozman, F., Moral, S., Walley, P. (eds.): Proceedings of the First International Symposium on Imprecise Probabilities and Their Applications (SIPTA), Universiteit Gent, Ghent, Belgium (1999)Google Scholar
  26. 26.
    de Cooman, G., Fine, T.L., Seidenfeld, T.: Proceedings of the 2nd International SIPTA. Shaker Publishing, The Netherlands (2001)Google Scholar
  27. 27.
    Bernard, J.M., Seidenfeld, T., Zaffalon, M. (eds.): Proceedings of the 3rd International SIPTA Carleton Scientific, Lugano, Switzerland (2003)Google Scholar
  28. 28.
    Cozman, F.G., Nau, R., Seidenfeld, T.: Proceedings of the Fourth International Symposium on Imprecise Probabilities and Their Applications. SIPTA (2005)Google Scholar
  29. 29.
    White III, C.C., Eldeib, H.K.: Markov decision processes with imprecise transition probabilities. Operations Research 42(4), 739–749 (1994)MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Satia, J.K., Lave Jr., R.E.: Markovian decision processes with uncertain transition probabilities. Operations Research 21(3), 728–740 (1973)MATHCrossRefMathSciNetGoogle Scholar
  31. 31.
    Howard, R.A.: Dynamic Porgramming and Markov Processes. MIT Press, Cambridge (1960)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Felipe W. Trevizan
    • 1
  • Fábio G. Cozman
    • 2
  • Leliane N. de Barros
    • 1
  1. 1.Instituto de Matemática e EstatísticaUniversidade de São PauloSão PauloBrazil
  2. 2.Escola PolitécnicaUniversidade de São PauloSão PauloBrazil

Personalised recommendations