Neural Networks and Soft Computing pp 626-631 | Cite as

# Solving Stochastic Shortest Path Problem Using Monte Carlo Sampling Method: A Distributed Learning Automata Approach

## Abstract

In this paper, we introduce a Monte Carlo simulation method based on distributed learning automata (DLA) for solving the stochastic shortest path problem. We give an iterative stochastic algorithm that finds the minimum expected value of set of random variables representing cost of paths in a stochastic graph by taking sufficient samples from them. In the given algorithm, the sample size is determined dynamically as the algorithm proceeds. It is shown that when the total sample size tends to infinity, the proposed algorithm finds the shortest path. In this algorithm, at each instant, DLA determine which edges to be sampled. This reduces the unnecessary sampling from the edges which don’t seem to be on the shortest path and thus reduces the overall sampling size. A new method of proof (different from [2,3]) is used to prove the convergence of the proposed algorithm. The simulations conducted confirm the theory.

## Preview

Unable to display preview. Download preview PDF.

### References

- 1.Polychronopoulos, G. H. and Tsitsiklis, J. N. (1996): Stochastic Shortest Path Problems with Recourse. Newtorks, 27, 133–143MathSciNetMATHGoogle Scholar
- 2.Meybodi, M. R. and Beigy, H. (2001): Solving Stochastic Shortest Path Problem Using Distributed Learning Automata. Proc. of 6th Annual Int. Computer Society of Iran Computer Conference CSICC-2001, Iran, 70–86Google Scholar
- 3.Beigy, H. and Meybodi, M. R. (2001): A New Distributed Learning Automata Based Algorithm For Solving Stochastic Shortest Path Problem, Tech. Rep. TRCE-2001–006, Computer Eng. Dept., Amirkabir Univ. of Tech., Tehran, IranGoogle Scholar
- 4.K. S. Narendra and K. S. Thathachar (1989): Learning Automata: An Introduction, New York: Printice-Hall.Google Scholar
- 5.Najim, K. and Pozyak, A. S. (1994): Learning Automata: Theory and Applications, Oxford: Pergamon pressGoogle Scholar
- 6.Meybodi, M. R. and Beigy, H. (2001): A Sampling Method Based on Distributed Learning Automata for Stochastic Shortest Path Problem, Tech. Rep. TR-CE2001–007, Computer Eng. Dept., Amirkabir Univ. of Tech., Tehran, IranGoogle Scholar
- 7.Alexopoulos, C. (1997): State Space Partitioning Methods for Stochastic Shortest Path Problems, Newtorks, 30, 9–21MathSciNetMATHGoogle Scholar