Skip to main content
SpringerLink
Log in

Solving Uncertain Markov Decision Problems: An Interval-Based Method

  • Conference paper
Book cover Advances in Natural Computation (ICNC 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4222))

Included in the following conference series:

Abstract

Stochastic Shortest Path problems (SSPs), a subclass of Markov Decision Problems (MDPs), can be efficiently dealt with VI, PI, RTDP, LAO* and so on. However, in many practical problems the estimation of the probabilities is far from accurate. In this paper, we present uncertain transition probabilities as close real intervals. Also, we describe a general algorithm, called gLAO*, that can solve uncertain MDPs efficiently. We demonstrate that Buffet and Aberdeen’s approach, searching for the best policy under the worst model, is a special case of our approaches. Experiments show that gLAO* inherits excellent performance of LAO* for solving uncertain MDPs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hansen, E.A., Ziberstein, S.: LAO*: A heuristic search algorithm that finds solutions with loops. Artificial Intelligence 129, 35–62 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  2. Bagnell, J.A., Ng, A.Y., Schneider, J.: Solving uncertain markov decision problems. Technical Report CMU-RI-TR-01-25, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (August 2001)

    Google Scholar 

  3. Daram, U.K., Chong, E.K.P., Shroff, N.B.: Markov Decision Processes with Uncertain Transition Rates: Sensitivity and Robust Control. In: Proceedings of the 41st IEEE, Conference on Devision and Control, Las Vegas, Nevada, USA (December 2002)

    Google Scholar 

  4. Buffet, O., Aberdeen, D.: Robust planning with (l)rtdp. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005) (2005)

    Google Scholar 

  5. Givan, R., Leach, S., Dean, T.: Bounded parameter markov decision processes. Artificial Intelligence 122(1-2), 71–109 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  6. Bertsekas, D.P., Tsitsiklis, J.N.: Neurodynamic Programming. Athena Scientific, Belmont (1996)

    Google Scholar 

  7. Bertsekas, D.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (1995)

    MATH  Google Scholar 

  8. Martelli, A., Montanari, U.: Optimizing decision trees through heuristically guided search. Comm. ACM 21(12), 1025–1039 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  9. Barto, A.G., Bradtke, S., Singh, S.: Learning to act using real time dynamic programming. Artificial Intelligence 72 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Software, Jilin University, Changchun, 130012, China

    Shulin Cui

  2. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Jigui Sun, Minghao Yin & Shuai Lu

  3. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Shulin Cui, Jigui Sun, Minghao Yin & Shuai Lu

Authors
  1. Shulin Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jigui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghao Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuai Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Life Science Research Center, School of Electronic Engineering, Xidian University, 710071, Xi’an, Shaanxi, China

    Licheng Jiao

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, Block S1, Nanyang Avenue, 639798, Singapore

    Lipo Wang

  3. School of Electronic Engineering, Xidian Univ., P.O. Box, 710071, Xi’an, P.R. China

    Xinbo Gao

  4. College of Mathematics and Information Science, Hebei Normal University, 050016, Shijiazhuang, Hebei, P.R. China

    Jing Liu

  5. Multi-Agent Systems Lab,Department of Computer Science, University of Science and Technology of China, 230026, Hefei, China

    Feng Wu

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Cui, S., Sun, J., Yin, M., Lu, S. (2006). Solving Uncertain Markov Decision Problems: An Interval-Based Method. In: Jiao, L., Wang, L., Gao, X., Liu, J., Wu, F. (eds) Advances in Natural Computation. ICNC 2006. Lecture Notes in Computer Science, vol 4222. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11881223_120

Download citation

  • DOI: https://doi.org/10.1007/11881223_120

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-45907-1

  • Online ISBN: 978-3-540-45909-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation