An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs

  • Hamid BeigyEmail author
  • Mohammad Reza Meybodi


In this paper, we study the problem of finding the shortest path in stochastic graphs and propose an iterative algorithm for solving it. This algorithm is based on distributed learning automata (DLA), and its objective is to use a DLA for finding the shortest path from the given source node to the given destination node whose weight is minimal in expected sense. At each stage of this algorithm, DLA specifies edges needed to be sampled. We show that the given algorithm finds the shortest path with minimum expected weight in stochastic graphs with high probability which can be close to unity as much as possible. We compare the given algorithm with some distributed learning automata-based iterative algorithms, a particle swarm optimization-based algorithm, an ant colony-based algorithm, a Q-learning-based algorithm, and an actor–critic-based algorithm for finding the shortest path. Computer experiments show that the proposed algorithm requires fewer edge samples to find the shortest path than the previously introduced DLA-based algorithms.


Learning automata Stochastic shortest path Distributed learning automata 



The authors would like to thank anonymous reviewers for their time, valuable comments, constructive criticism, and suggestions which greatly improved the paper.


  1. 1.
    Adlakha VG (1986) An improved conditional Monte Carlo technique for the stochastic shortest route problem. Manag Sci 32(10):1360–1367zbMATHCrossRefGoogle Scholar
  2. 2.
    Alexopoulos C (1997) State Space partitioning methods for stochastic shortest path problems. Networks 30:9–21MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Beigy H, Meybodi MR (2000) Adaptation of Parameters of BP algorithm using learning automata. In: Proceedings of VI Brazilian Symposium on Neural Networks, SBRN2000, Brazil, pp 24–31Google Scholar
  4. 4.
    Beigy H, Meybodi MR (2000) Solving the graph isomorphism problem using learning automata. In: Proceedings of 5th Annual International Computer Society of Iran Computer Conference, CISCC-2000, Tehran, Iran, pp 402–415Google Scholar
  5. 5.
    Beigy H, Meybodi MR (2001) Backpropagation algorithm adaptation parameters using learning automata. Int J Neural Syst 11(3):219–228zbMATHCrossRefGoogle Scholar
  6. 6.
    Beigy H, Meybodi MR (2002) Call admission control in cellular mobile networks: a learning automata approach, vol 2510. Springer-Verlag lecture notes in computer science. Springer, Berlin, pp 450–457zbMATHGoogle Scholar
  7. 7.
    Beigy H, Meybodi MR (2005) An adaptive call admission algorithm for cellular networks. Electr Comput Eng 31(2):132–151zbMATHCrossRefGoogle Scholar
  8. 8.
    Beigy H, Meybodi MR (2006) Utilizing distributed learning automata to solve stochastic shortest path problems. Int J Uncertain Fuzziness Knowl Based Syst 14(5):591–615MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Beigy H, Meybodi MR (2009) Adaptive limited fractional guard channel algorithms: a learning automata approach. Int J Uncertain Fuzziness Knowl Based Syst 17(6):881–913CrossRefGoogle Scholar
  10. 10.
    Beigy H, Meybodi MR (2009) Cellular learning automata based dynamic channel assignment algorithms. Int J Comput Intell Appl 8(3):287–314zbMATHCrossRefGoogle Scholar
  11. 11.
    Beigy H, Meybodi MR (2009) A learning automata-based algorithm for determination of the number of hidden units for three layer neural networks. Int J Syst Sci 40(1):101–118zbMATHCrossRefGoogle Scholar
  12. 12.
    Chen BY, Shi C, Zhang J, Lam WHK, Li Q, Xiang S (2017) Most reliable path-finding algorithm for maximizing on-time arrival probability. Transp B Transp Dyn 5(3):1–17Google Scholar
  13. 13.
    Das A, Martel C (2009) Stochastic shortest path with unlimited hopes. Inf Process Lett 109:290–295zbMATHCrossRefGoogle Scholar
  14. 14.
    Deo N, Pang C (1984) Shortest path algorithms: taxonomy and annotation. Networks 14:275–303MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Engelbrecht AP (2007) Computational intelligence: an introduction. Wiley, HobokenCrossRefGoogle Scholar
  16. 16.
    Fan YY, Kalaba RE, Moore JE (2005) Shortest paths stochastic networks with correlated link costs. Comput Math Appl 49:1549–1564MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Frank H (1969) Shortest path in probabilistic graphs. Oper Res 15:583–599MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Fu L, Rilett LR (1998) Expected shortest paths in dynamic and stochastic traffic networks. Transp Res Part B Methodol 32:499–516CrossRefGoogle Scholar
  19. 19.
    Guillot M, Stauffer G (2019) The stochastic shortest path problem: a polyhedral combinatorics perspective. Eur J Oper Res (in press)Google Scholar
  20. 20.
    Hasanzadeh M, Meybodi MR (2014) Grid resource discovery based on distributed learning automata. Computing 96(9):909–922zbMATHCrossRefGoogle Scholar
  21. 21.
    Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of International Conference on Neural Networks (ICNN’95), pp 1942–1948Google Scholar
  22. 22.
    Lakshmivarahan S, Thathachar MAL (1976) Bounds on the convergence probabilities of learning automata. IEEE Trans Syst Man Cybern SMC–6:756–763MathSciNetzbMATHGoogle Scholar
  23. 23.
    Martin JJ (1965) Distribution of the time through a directed acyclic network. Oper Res 13:46–56MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Meybodi MR, Beigy H (2001) Neural network engineering using learning automata: determining of desired size of three layer feedforward neural networks. J Fac Eng 34(4):1–26Google Scholar
  25. 25.
    Meybodi MR, Beigy H (2002) A note on learning automata based schemes for adaptation of BP parameters. J Neurocomput 48(3):957–974zbMATHCrossRefGoogle Scholar
  26. 26.
    Meybodi MR, Beigy H (2002) New learning automata based algorithms for adaptation of backpropagation algorithm parameters. Int J Neural Syst 12(1):45–68zbMATHCrossRefGoogle Scholar
  27. 27.
    Meybodi MR, Laksmivarahan S (1983) A learning approach to priority assignment in a two class M/M/1 queueing with unknown parameters. In: Proceedings of the Third Yale Workshop on Applications of Adaptive Systems Theory, pp 106–109Google Scholar
  28. 28.
    Meybodi MRM, Meybodi MR (2014) Extended distributed learning automata: an automata-based framework for solving stochastic graph optimization problems. Appl Intell 41(3):923–940CrossRefGoogle Scholar
  29. 29.
    Miller-Hooks E (2001) Adaptive least-expected time paths in stochastic, time-varying transportation and data networks. Networks 37(1):35–52MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Miller-Hooks E, Mahmassani H (2000) Least expected time paths in stochastic, time-varying transportation networks. Transp Sci 34:198–215zbMATHCrossRefGoogle Scholar
  31. 31.
    Mirchandani PB (1970) Shortest distance and reliability of probabilistic networks. Comput Oper Res 8:347–355Google Scholar
  32. 32.
    Mirchandani PB, Soroush H (1985) Shortest distance and reliability of probabilistic networks: a case with temporary preferences. Comput Oper Res 13(4):365-087zbMATHGoogle Scholar
  33. 33.
    Momtazi S, Kafi S, Beigy H (2008) Solving stochastic path problem: particle swarm optimization approach, vol 5027. Springer-Verlag lecture notes in artificial intelligence, Springer, Berlin, pp 590–600Google Scholar
  34. 34.
    Moradabadi B, Beigy H (2014) A new real-coded Bayesian optimization algorithm based on a team of learning automata for continuous optimization. Gen Program Evol Mach 15(2):169–193CrossRefGoogle Scholar
  35. 35.
    Moradabadi B, Meybodi MR (2017) Link prediction in fuzzy social networks using distributed learning automata. Appl Intell 47(3):837–849CrossRefGoogle Scholar
  36. 36.
    Moradabadi B, Meybodi MR (2017) Link prediction in stochastic social networks: learning automata approach. J Comput Sci 127:313–328MathSciNetGoogle Scholar
  37. 37.
    Narendra KS, Thathachar KS (1989) Learning automata: an introduction. Printice-Hall, New YorkGoogle Scholar
  38. 38.
    Nedzelnitsky OV, Narendra KS (1987) Nonstationary models of learning automata routing in data communication networks. IEEE Trans Syst Man Cybern SMC–17(6):1004–1015CrossRefGoogle Scholar
  39. 39.
    Oommen BJ, Roberts TD (2000) Continuous learning automata solutions to the capacity assignment problem. IEEE Trans Comput 49(6):608–620CrossRefGoogle Scholar
  40. 40.
    Oommen BJ, de St. Croix EV (1996) Graph partitioning using learning automata. IEEE Trans Comput 45(2):195–208MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Peer SK, Sharma DK (2007) Finding the shortest path in stochastic networks. Comput Math Appl 53:729–740MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Pineda LE, Wray KH, Zilberstein S (2017) Fast SSP solvers using short-sighted labeling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4–9, 2017, San Francisco, California, USA, pp 3629–3635Google Scholar
  43. 43.
    Polychronopoulos GH, Tsitsiklis JN (1990) Stochastic shortest path problems with recourse. Networks 27:133–443MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Pritsker AAB (1966) Application of multi-channel queuing results to the analysis of conveyor systems. J Ind Eng 17:14–42Google Scholar
  45. 45.
    Rezvanian A, Meybodi MR (2015) Finding maximum clique in stochastic graphs using distributed learning automata. Int J Uncertain Fuzziness Knowl Based Syst 23(1):1–32MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Saghiri AM, Khomami MD, Meybodi MR (2019) Intelligent random walk: an approach based on learning automata. Springer, BerlinCrossRefGoogle Scholar
  47. 47.
    Sigal CC, Pritsker AAB, Solberg JJ (1980) The stochastic shortest route problem. Oper Res 48:1122–1129MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Sizal CU, Pritsker VAQ, Solberg JJ (1979) The use of cutsets in Monte-Carlo analysis of stochastic networks. Math Comput Simul 21:276–384MathSciNetCrossRefGoogle Scholar
  49. 49.
    Srikantakumar PR, Narendra KS (1982) A learning model for routing in telephone networks. SIAM J Control Optim 20(1):34–57MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgezbMATHGoogle Scholar
  51. 51.
    Teichteil-Königsbuch F (2012) Stochastic safest and shortest path problems. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, pp 1825–1831Google Scholar
  52. 52.
    Thathachar MAL, Ramakrishnan KR (1981) A hierarchical system of learning automata. IEEE Trans Syst Man Cybern SMC–11(3):236–248MathSciNetzbMATHGoogle Scholar
  53. 53.
    Thomas B, White C (2007) The dynamic shortest path problem with anticipation. Eur J Oper Res 176(2):836–854MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Vahidipour S, Esnaashari M, Rezvanian A, Meybodi M (2019) Gapn-la: a framework for solving graph problems using petri nets and learning automata. Eng Appl Artif Intell 77:255–267CrossRefGoogle Scholar
  55. 55.
    Waller ST, Ziliaskopoulos AK (2002) On the online stochastic shortest paths with limited arc cost dependencies. Networks 40(4):216–227MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Zeng W, Miwa T, Wakita Y, Morikawa T (2015) Application of lagrangian relaxation approach to \(\alpha\)-reliable path finding in stochastic networks with correlated link travel times. Transp Res Part C Emerg 56(3):309–334CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer EngineeringSharif University of TechnologyTehranIran
  2. 2.Department of Computer EngineeringAmirkabir University of TechnologyTehranIran

Personalised recommendations