\(\delta \)-Radius Unified Influence Value Reinforcement Learning

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 474)

Abstract

Nowadays Decentralized Partial Observable Markov Decision Process framework represents the actual state of art in Multi-Agent System. Dec-POMDP incorporates the concepts of independent view and message exchange to the original POMDP model, opening new possibilities about the independent views for each agent in the system. Nevertheless there are some limitations about the communication.

About communication on MAS, Dec-POMDP is still focused in the message structure and content instead of the communication relationship between agents, which is our focus. On the other hand, the convergence on MAS is about the group of agents convergence as a whole, to achieve it the collaboration between the agents is necessary.

The collaboration and/or communication cost in MAS is high, in computational cost terms, to improve this is important to limit the communication between agents to the only necessary cases.

The present approach is focused in the impact of the communication limitation on MAS, and how it may improve the use of system resources, by reducing computational, without harming the global convergence. In this sense \(\delta \)-radius is a unified algorithm, based on Influence Value Reinforcement Learning and Independent Learning models, that allows restriction of the communication by the variation of \(\delta \).

Keywords

Multi-Agent Systems Artificial Intelligence Markov Decision Process 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amato, C., Chowdhary, G., Geramifard, A., Ure, N., Kochenderfer, M.: Decentralized control of partially observable markov decision processes. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 2398–2405, December 2013Google Scholar
  2. 2.
    Barrios-Aranibar, D., Gonçalves, L.M.G.: Learning from delayed rewards using influence values applied to coordination in multi-agent systems. In: VIII SBAI-Simpósio Brasileiro de Automaç ao Inteligente (2007)Google Scholar
  3. 3.
    Barrios Aranibar, D., Gonçalves, L.M.G., de Carvalho, F.V.: Aprendizado por Reforço com Valores de Influência em Sistemas Multi-Agente (2009)Google Scholar
  4. 4.
    Goldman, C.V., Zilberstein, S.: Optimizing information exchange in cooperative multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2003, pp. 137–144. ACM, New York (2003)Google Scholar
  5. 5.
    Guestrin, C., Venkataraman, S., Koller, D.: Context-specific multiagent coordination and planning with factored mdps. In: Proceedings of the Eighteenth National Conference on Artificial Intelligence and Fourteenth Conference on Innovative Applications of Artificial Intelligence, July 28 - August 1, 2002, Edmonton, Alberta, Canada, pp. 253–259 (2002)Google Scholar
  6. 6.
    Pini, G., Gagliolo, M., Brutschy, A., Dorigo, M., Birattari, M.: Task partitioning in a robot swarm: a study on the effect of communication. Swarm Intelligence 7(2), 173–199 (2013)CrossRefGoogle Scholar
  7. 7.
    Tan, M.: Multi-agent reinforcement learning: independent versus cooperative agents. In: Proceedings of the Tenth International Conference on Machine Learning (ICML 1993), pp. 330–337. Morgan Kauffman, San Francisco (1993)Google Scholar
  8. 8.
    Whitehead, S.D.: A complexity analysis of cooperative mechanisms in reinforcement learning. In: Proceedings of AAAI 1991, Anaheim, CA, pp. 607–613 (1991)Google Scholar
  9. 9.
    Zhang, C., Lesser, V.: Coordinating multi-agent reinforcement learning with limited communication. In: Ito, J., Gini, S. (eds.) Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, IFAAMAS, St. Paul, MN, pp. 1101–1108 (2013)Google Scholar
  10. 10.
    Zhang, K., Maeda, Y., Takahashi, Y.: Group behavior learning in multi-agent systems based on social interaction among agents. SCIS & ISIS 12010, 193–198 (2010)Google Scholar
  11. 11.
    Åström, K.: Optimal control of markov processes with incomplete state information. Journal of Mathematical Analysis and Applications 10(1), 174–205 (1965)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • J. Alejandro Camargo
    • 1
  • Dennis Barrios-Aranibar
    • 1
  1. 1.San Pablo Catholic UniversityArequipaPeru

Personalised recommendations