Solving Stochastic Shortest Path Problem Using Monte Carlo Sampling Method: A Distributed Learning Automata Approach

  • M. R. Meybodi
  • Hamid Beigy
Conference paper
Part of the Advances in Soft Computing book series (AINSC, volume 19)

Abstract

In this paper, we introduce a Monte Carlo simulation method based on distributed learning automata (DLA) for solving the stochastic shortest path problem. We give an iterative stochastic algorithm that finds the minimum expected value of set of random variables representing cost of paths in a stochastic graph by taking sufficient samples from them. In the given algorithm, the sample size is determined dynamically as the algorithm proceeds. It is shown that when the total sample size tends to infinity, the proposed algorithm finds the shortest path. In this algorithm, at each instant, DLA determine which edges to be sampled. This reduces the unnecessary sampling from the edges which don’t seem to be on the shortest path and thus reduces the overall sampling size. A new method of proof (different from [2,3]) is used to prove the convergence of the proposed algorithm. The simulations conducted confirm the theory.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Polychronopoulos, G. H. and Tsitsiklis, J. N. (1996): Stochastic Shortest Path Problems with Recourse. Newtorks, 27, 133–143MathSciNetMATHGoogle Scholar
  2. 2.
    Meybodi, M. R. and Beigy, H. (2001): Solving Stochastic Shortest Path Problem Using Distributed Learning Automata. Proc. of 6th Annual Int. Computer Society of Iran Computer Conference CSICC-2001, Iran, 70–86Google Scholar
  3. 3.
    Beigy, H. and Meybodi, M. R. (2001): A New Distributed Learning Automata Based Algorithm For Solving Stochastic Shortest Path Problem, Tech. Rep. TRCE-2001–006, Computer Eng. Dept., Amirkabir Univ. of Tech., Tehran, IranGoogle Scholar
  4. 4.
    K. S. Narendra and K. S. Thathachar (1989): Learning Automata: An Introduction, New York: Printice-Hall.Google Scholar
  5. 5.
    Najim, K. and Pozyak, A. S. (1994): Learning Automata: Theory and Applications, Oxford: Pergamon pressGoogle Scholar
  6. 6.
    Meybodi, M. R. and Beigy, H. (2001): A Sampling Method Based on Distributed Learning Automata for Stochastic Shortest Path Problem, Tech. Rep. TR-CE2001–007, Computer Eng. Dept., Amirkabir Univ. of Tech., Tehran, IranGoogle Scholar
  7. 7.
    Alexopoulos, C. (1997): State Space Partitioning Methods for Stochastic Shortest Path Problems, Newtorks, 30, 9–21MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • M. R. Meybodi
    • 1
  • Hamid Beigy
    • 1
  1. 1.Soft Computing Laboratory Computer Engineering DepartmentAmirkabir University of TechnologyTehranIran

Personalised recommendations