Skip to main content
SpringerLink
Log in

An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In this paper, we study the problem of finding the shortest path in stochastic graphs and propose an iterative algorithm for solving it. This algorithm is based on distributed learning automata (DLA), and its objective is to use a DLA for finding the shortest path from the given source node to the given destination node whose weight is minimal in expected sense. At each stage of this algorithm, DLA specifies edges needed to be sampled. We show that the given algorithm finds the shortest path with minimum expected weight in stochastic graphs with high probability which can be close to unity as much as possible. We compare the given algorithm with some distributed learning automata-based iterative algorithms, a particle swarm optimization-based algorithm, an ant colony-based algorithm, a Q-learning-based algorithm, and an actor–critic-based algorithm for finding the shortest path. Computer experiments show that the proposed algorithm requires fewer edge samples to find the shortest path than the previously introduced DLA-based algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Adlakha VG (1986) An improved conditional Monte Carlo technique for the stochastic shortest route problem. Manag Sci 32(10):1360–1367

    Article  MATH  Google Scholar 

  2. Alexopoulos C (1997) State Space partitioning methods for stochastic shortest path problems. Networks 30:9–21

    Article  MathSciNet  MATH  Google Scholar 

  3. Beigy H, Meybodi MR (2000) Adaptation of Parameters of BP algorithm using learning automata. In: Proceedings of VI Brazilian Symposium on Neural Networks, SBRN2000, Brazil, pp 24–31

  4. Beigy H, Meybodi MR (2000) Solving the graph isomorphism problem using learning automata. In: Proceedings of 5th Annual International Computer Society of Iran Computer Conference, CISCC-2000, Tehran, Iran, pp 402–415

  5. Beigy H, Meybodi MR (2001) Backpropagation algorithm adaptation parameters using learning automata. Int J Neural Syst 11(3):219–228

    Article  MATH  Google Scholar 

  6. Beigy H, Meybodi MR (2002) Call admission control in cellular mobile networks: a learning automata approach, vol 2510. Springer-Verlag lecture notes in computer science. Springer, Berlin, pp 450–457

    MATH  Google Scholar 

  7. Beigy H, Meybodi MR (2005) An adaptive call admission algorithm for cellular networks. Electr Comput Eng 31(2):132–151

    Article  MATH  Google Scholar 

  8. Beigy H, Meybodi MR (2006) Utilizing distributed learning automata to solve stochastic shortest path problems. Int J Uncertain Fuzziness Knowl Based Syst 14(5):591–615

    Article  MathSciNet  MATH  Google Scholar 

  9. Beigy H, Meybodi MR (2009) Adaptive limited fractional guard channel algorithms: a learning automata approach. Int J Uncertain Fuzziness Knowl Based Syst 17(6):881–913

    Article  Google Scholar 

  10. Beigy H, Meybodi MR (2009) Cellular learning automata based dynamic channel assignment algorithms. Int J Comput Intell Appl 8(3):287–314

    Article  MATH  Google Scholar 

  11. Beigy H, Meybodi MR (2009) A learning automata-based algorithm for determination of the number of hidden units for three layer neural networks. Int J Syst Sci 40(1):101–118

    Article  MATH  Google Scholar 

  12. Chen BY, Shi C, Zhang J, Lam WHK, Li Q, Xiang S (2017) Most reliable path-finding algorithm for maximizing on-time arrival probability. Transp B Transp Dyn 5(3):1–17

    Google Scholar 

  13. Das A, Martel C (2009) Stochastic shortest path with unlimited hopes. Inf Process Lett 109:290–295

    Article  MATH  Google Scholar 

  14. Deo N, Pang C (1984) Shortest path algorithms: taxonomy and annotation. Networks 14:275–303

    Article  MathSciNet  MATH  Google Scholar 

  15. Engelbrecht AP (2007) Computational intelligence: an introduction. Wiley, Hoboken

    Book  Google Scholar 

  16. Fan YY, Kalaba RE, Moore JE (2005) Shortest paths stochastic networks with correlated link costs. Comput Math Appl 49:1549–1564

    Article  MathSciNet  MATH  Google Scholar 

  17. Frank H (1969) Shortest path in probabilistic graphs. Oper Res 15:583–599

    Article  MathSciNet  MATH  Google Scholar 

  18. Fu L, Rilett LR (1998) Expected shortest paths in dynamic and stochastic traffic networks. Transp Res Part B Methodol 32:499–516

    Article  Google Scholar 

  19. Guillot M, Stauffer G (2019) The stochastic shortest path problem: a polyhedral combinatorics perspective. Eur J Oper Res (in press)

  20. Hasanzadeh M, Meybodi MR (2014) Grid resource discovery based on distributed learning automata. Computing 96(9):909–922

    Article  MATH  Google Scholar 

  21. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of International Conference on Neural Networks (ICNN’95), pp 1942–1948

  22. Lakshmivarahan S, Thathachar MAL (1976) Bounds on the convergence probabilities of learning automata. IEEE Trans Syst Man Cybern SMC–6:756–763

    MathSciNet  MATH  Google Scholar 

  23. Martin JJ (1965) Distribution of the time through a directed acyclic network. Oper Res 13:46–56

    Article  MathSciNet  MATH  Google Scholar 

  24. Meybodi MR, Beigy H (2001) Neural network engineering using learning automata: determining of desired size of three layer feedforward neural networks. J Fac Eng 34(4):1–26

    Google Scholar 

  25. Meybodi MR, Beigy H (2002) A note on learning automata based schemes for adaptation of BP parameters. J Neurocomput 48(3):957–974

    Article  MATH  Google Scholar 

  26. Meybodi MR, Beigy H (2002) New learning automata based algorithms for adaptation of backpropagation algorithm parameters. Int J Neural Syst 12(1):45–68

    Article  MATH  Google Scholar 

  27. Meybodi MR, Laksmivarahan S (1983) A learning approach to priority assignment in a two class M/M/1 queueing with unknown parameters. In: Proceedings of the Third Yale Workshop on Applications of Adaptive Systems Theory, pp 106–109

  28. Meybodi MRM, Meybodi MR (2014) Extended distributed learning automata: an automata-based framework for solving stochastic graph optimization problems. Appl Intell 41(3):923–940

    Article  Google Scholar 

  29. Miller-Hooks E (2001) Adaptive least-expected time paths in stochastic, time-varying transportation and data networks. Networks 37(1):35–52

    Article  MathSciNet  MATH  Google Scholar 

  30. Miller-Hooks E, Mahmassani H (2000) Least expected time paths in stochastic, time-varying transportation networks. Transp Sci 34:198–215

    Article  MATH  Google Scholar 

  31. Mirchandani PB (1970) Shortest distance and reliability of probabilistic networks. Comput Oper Res 8:347–355

    Google Scholar 

  32. Mirchandani PB, Soroush H (1985) Shortest distance and reliability of probabilistic networks: a case with temporary preferences. Comput Oper Res 13(4):365-087

    MATH  Google Scholar 

  33. Momtazi S, Kafi S, Beigy H (2008) Solving stochastic path problem: particle swarm optimization approach, vol 5027. Springer-Verlag lecture notes in artificial intelligence, Springer, Berlin, pp 590–600

    Google Scholar 

  34. Moradabadi B, Beigy H (2014) A new real-coded Bayesian optimization algorithm based on a team of learning automata for continuous optimization. Gen Program Evol Mach 15(2):169–193

    Article  Google Scholar 

  35. Moradabadi B, Meybodi MR (2017) Link prediction in fuzzy social networks using distributed learning automata. Appl Intell 47(3):837–849

    Article  Google Scholar 

  36. Moradabadi B, Meybodi MR (2017) Link prediction in stochastic social networks: learning automata approach. J Comput Sci 127:313–328

    MathSciNet  Google Scholar 

  37. Narendra KS, Thathachar KS (1989) Learning automata: an introduction. Printice-Hall, New York

    Google Scholar 

  38. Nedzelnitsky OV, Narendra KS (1987) Nonstationary models of learning automata routing in data communication networks. IEEE Trans Syst Man Cybern SMC–17(6):1004–1015

    Article  Google Scholar 

  39. Oommen BJ, Roberts TD (2000) Continuous learning automata solutions to the capacity assignment problem. IEEE Trans Comput 49(6):608–620

    Article  Google Scholar 

  40. Oommen BJ, de St. Croix EV (1996) Graph partitioning using learning automata. IEEE Trans Comput 45(2):195–208

    Article  MathSciNet  MATH  Google Scholar 

  41. Peer SK, Sharma DK (2007) Finding the shortest path in stochastic networks. Comput Math Appl 53:729–740

    Article  MathSciNet  MATH  Google Scholar 

  42. Pineda LE, Wray KH, Zilberstein S (2017) Fast SSP solvers using short-sighted labeling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4–9, 2017, San Francisco, California, USA, pp 3629–3635

  43. Polychronopoulos GH, Tsitsiklis JN (1990) Stochastic shortest path problems with recourse. Networks 27:133–443

    Article  MathSciNet  MATH  Google Scholar 

  44. Pritsker AAB (1966) Application of multi-channel queuing results to the analysis of conveyor systems. J Ind Eng 17:14–42

    Google Scholar 

  45. Rezvanian A, Meybodi MR (2015) Finding maximum clique in stochastic graphs using distributed learning automata. Int J Uncertain Fuzziness Knowl Based Syst 23(1):1–32

    Article  MathSciNet  MATH  Google Scholar 

  46. Saghiri AM, Khomami MD, Meybodi MR (2019) Intelligent random walk: an approach based on learning automata. Springer, Berlin

    Book  Google Scholar 

  47. Sigal CC, Pritsker AAB, Solberg JJ (1980) The stochastic shortest route problem. Oper Res 48:1122–1129

    Article  MathSciNet  MATH  Google Scholar 

  48. Sizal CU, Pritsker VAQ, Solberg JJ (1979) The use of cutsets in Monte-Carlo analysis of stochastic networks. Math Comput Simul 21:276–384

    Article  MathSciNet  Google Scholar 

  49. Srikantakumar PR, Narendra KS (1982) A learning model for routing in telephone networks. SIAM J Control Optim 20(1):34–57

    Article  MathSciNet  MATH  Google Scholar 

  50. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    MATH  Google Scholar 

  51. Teichteil-Königsbuch F (2012) Stochastic safest and shortest path problems. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, pp 1825–1831

  52. Thathachar MAL, Ramakrishnan KR (1981) A hierarchical system of learning automata. IEEE Trans Syst Man Cybern SMC–11(3):236–248

    MathSciNet  MATH  Google Scholar 

  53. Thomas B, White C (2007) The dynamic shortest path problem with anticipation. Eur J Oper Res 176(2):836–854

    Article  MathSciNet  MATH  Google Scholar 

  54. Vahidipour S, Esnaashari M, Rezvanian A, Meybodi M (2019) Gapn-la: a framework for solving graph problems using petri nets and learning automata. Eng Appl Artif Intell 77:255–267

    Article  Google Scholar 

  55. Waller ST, Ziliaskopoulos AK (2002) On the online stochastic shortest paths with limited arc cost dependencies. Networks 40(4):216–227

    Article  MathSciNet  MATH  Google Scholar 

  56. Zeng W, Miwa T, Wakita Y, Morikawa T (2015) Application of lagrangian relaxation approach to \(\alpha\)-reliable path finding in stochastic networks with correlated link travel times. Transp Res Part C Emerg 56(3):309–334

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank anonymous reviewers for their time, valuable comments, constructive criticism, and suggestions which greatly improved the paper.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Sharif University of Technology, Tehran, Iran

    Hamid Beigy

  2. Department of Computer Engineering, Amirkabir University of Technology, Tehran, Iran

    Mohammad Reza Meybodi

Authors
  1. Hamid Beigy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Reza Meybodi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Beigy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beigy, H., Meybodi, M.R. An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs. J Supercomput 76, 5540–5562 (2020). https://doi.org/10.1007/s11227-019-03085-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-03085-0

Keywords

Search

Navigation