Advertisement

An Estimator Update Scheme for Large Teams of Learning Automata

  • Manuel P. Cuéllar
  • María Ros
  • Miguel Delgado
  • Amparo Vila
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 91)

Abstract

Learning Automata are stochastic decision-making machines that have been widely used in classification, control, and network routing, between others. Despite their versatility, one of the main drawbacks of these models is the low convergence rate of the learning rules used for the training. Estimator algorithms such as Pursuit schemes help to overcome this limitation, although they require a high computer memory cost for their operation. This fact becomes a serious inconvenient when a large set of learning automata collaborate in a team to solve a concrete task, since the memory requirements of these algorithms increases exponentially. In these cases, Pursuit algorithms are ineffective due to memory overflow.

In this work, we address this problem and we propose an estimator algorithm that can be used to train large teams of Learning Automata. The approach uses a similar strategy to Tabu Search algorithms to manage long and short term memory, in order to reduce the memory requirements. The method is applied in classic permutation problems as a test-bed.

Keywords

Estimator Algorithm Term Memory Quadratic Assignment Problem Learn Automaton Learn Automaton 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agache, M., Oommen, B.J.: Generalized pursuit learning schemes: new families of continuous and discretized learning automata. IEEE Transactions on Systems, Man, and Cybernetics, Part B 32(6), 738–749 (2002)CrossRefGoogle Scholar
  2. 2.
    Baba, N., Mogami, Y.: A new learning algorithm for the hierarchical structure learning automata operating in the nonstationary s-model random environment. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 32(6), 750–758 (2002)CrossRefGoogle Scholar
  3. 3.
    Baba, N., Mogami, Y.: A relative reward-strength algorithm for the hierarchical structure learning automata operating in the general nonstationary multiteacher environment. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36(4), 781–794 (2006)CrossRefGoogle Scholar
  4. 4.
    Beigy, H., Meybodi, M.R.: Utilizing distributed learning automata to solve stochastic shortest path problems. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 14(5), 591–615 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Glover, F., Laguna, M.: Tabu search, pp. 70–150 (1993)Google Scholar
  6. 6.
    Lukac, K., Lukac, Z., Tkalic, M.: Behaviour of f learning automata as multicriteria routing agents in connection oriented networks. In: The 12th IEEE International Conference on Fuzzy Systems, FUZZ 2003, vol. 1, pp. 296–301 (2003)Google Scholar
  7. 7.
    Narendra, K.S., Thathachar, M.A.L.: Learning automata: an introduction. Prentice-Hall, Inc., Upper Saddle River (1989)Google Scholar
  8. 8.
    Nowé, A., Verbeeck, K., Peeters, M.: Learning automata as a basis for multi agent reinforcement learning. In: Tuyls, K., ’t Hoen, P.J., Verbeeck, K., Sen, S. (eds.) LAMAS 2005. LNCS (LNAI), vol. 3898, pp. 71–85. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Oommen, B., Agache, M.: A comparison of continuous and discretized pursuit learning schemes. In: Proceedings of 1999 IEEE International Conference on Systems, Man, and Cybernetics, SMC 1999, vol. 4, pp. 1061–1067 (1999)Google Scholar
  10. 10.
    Oommen, B., Lanctot, J.: Discretized pursuit learning automata. IEEE Transactions on Systems, Man and Cybernetics 20(4), 931–938 (1990)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Oommen, B.J., Hashem, M.K.: Modeling a student-classroom interaction in a tutorial-like system using learning automata. Trans. Sys. Man Cyber. Part B 40(1), 29–42 (2010), http://dx.doi.org/10.1109/TSMCB.2009.2032414 CrossRefGoogle Scholar
  12. 12.
    Peeters, M., Verbeeck, K., Nowé, A.: Solving multi-stage games with hierarchical learning automata that bootstrap. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds.) ALAMAS 2005, ALAMAS 2006, and ALAMAS 2007. LNCS (LNAI), vol. 4865, pp. 169–187. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Poznyak, A.S., Najim, K.: Learning Automata and Stochastic Optimization. Springer-Verlag New York, Inc., Secaucus (1997)zbMATHGoogle Scholar
  14. 14.
    Sastry, P., Thathachar, M.: Learning automata algorithms for pattern classification. Sadhana 24(4-5), 261–292 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Thathachar, M.A.L., Arvind, M.T.: Parallel algorithms for modules of learning automata. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(1), 24–33 (1998)CrossRefGoogle Scholar
  16. 16.
    Thathachar, M.A.L., Ramakrishnan, K.R.: A cooperative game of a pair of learning automata. Automatica 20(6), 797–801 (1984)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Thathachar, M.A.L., Sastry, P.S.: Varieties of learning automata: an overview. IEEE Transactions on Systems, Man, and Cybernetics, Part B 32(6), 711–722 (2002)CrossRefGoogle Scholar
  18. 18.
    Torkestani, J., Meybodi, M.: Graph coloring problem based on learning automata. In: International Conference on Information Management and Engineering, ICIME 2009, pp. 718–722 (2009)Google Scholar
  19. 19.
    Torkestani, J., Meybodi, M.: Solving the minimum spanning tree problem in stochastic graphs using learning automata. In: International Conference on Information Management and Engineering, ICIME 2009, pp. 643–647 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Manuel P. Cuéllar
    • 1
  • María Ros
    • 1
  • Miguel Delgado
    • 1
  • Amparo Vila
    • 1
  1. 1.Department of Computer Science and Artificial Intelligence E.T.S.I. Informática y de TelecomunicaciónUniversity of GranadaSpain

Personalised recommendations