Skip to main content

\(\delta \)-Radius Unified Influence Value Reinforcement Learning

  • 1456 Accesses

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 474)

Abstract

Nowadays Decentralized Partial Observable Markov Decision Process framework represents the actual state of art in Multi-Agent System. Dec-POMDP incorporates the concepts of independent view and message exchange to the original POMDP model, opening new possibilities about the independent views for each agent in the system. Nevertheless there are some limitations about the communication.

About communication on MAS, Dec-POMDP is still focused in the message structure and content instead of the communication relationship between agents, which is our focus. On the other hand, the convergence on MAS is about the group of agents convergence as a whole, to achieve it the collaboration between the agents is necessary.

The collaboration and/or communication cost in MAS is high, in computational cost terms, to improve this is important to limit the communication between agents to the only necessary cases.

The present approach is focused in the impact of the communication limitation on MAS, and how it may improve the use of system resources, by reducing computational, without harming the global convergence. In this sense \(\delta \)-radius is a unified algorithm, based on Influence Value Reinforcement Learning and Independent Learning models, that allows restriction of the communication by the variation of \(\delta \).

Keywords

  • Multi-Agent Systems
  • Artificial Intelligence
  • Markov Decision Process

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-40162-1_14
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-40162-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amato, C., Chowdhary, G., Geramifard, A., Ure, N., Kochenderfer, M.: Decentralized control of partially observable markov decision processes. In: 2013 IEEE 52nd Annual Conference on Decision and Control (CDC), pp. 2398–2405, December 2013

    Google Scholar 

  2. Barrios-Aranibar, D., Gonçalves, L.M.G.: Learning from delayed rewards using influence values applied to coordination in multi-agent systems. In: VIII SBAI-Simpósio Brasileiro de Automaç ao Inteligente (2007)

    Google Scholar 

  3. Barrios Aranibar, D., Gonçalves, L.M.G., de Carvalho, F.V.: Aprendizado por Reforço com Valores de Influência em Sistemas Multi-Agente (2009)

    Google Scholar 

  4. Goldman, C.V., Zilberstein, S.: Optimizing information exchange in cooperative multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2003, pp. 137–144. ACM, New York (2003)

    Google Scholar 

  5. Guestrin, C., Venkataraman, S., Koller, D.: Context-specific multiagent coordination and planning with factored mdps. In: Proceedings of the Eighteenth National Conference on Artificial Intelligence and Fourteenth Conference on Innovative Applications of Artificial Intelligence, July 28 - August 1, 2002, Edmonton, Alberta, Canada, pp. 253–259 (2002)

    Google Scholar 

  6. Pini, G., Gagliolo, M., Brutschy, A., Dorigo, M., Birattari, M.: Task partitioning in a robot swarm: a study on the effect of communication. Swarm Intelligence 7(2), 173–199 (2013)

    CrossRef  Google Scholar 

  7. Tan, M.: Multi-agent reinforcement learning: independent versus cooperative agents. In: Proceedings of the Tenth International Conference on Machine Learning (ICML 1993), pp. 330–337. Morgan Kauffman, San Francisco (1993)

    Google Scholar 

  8. Whitehead, S.D.: A complexity analysis of cooperative mechanisms in reinforcement learning. In: Proceedings of AAAI 1991, Anaheim, CA, pp. 607–613 (1991)

    Google Scholar 

  9. Zhang, C., Lesser, V.: Coordinating multi-agent reinforcement learning with limited communication. In: Ito, J., Gini, S. (eds.) Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, IFAAMAS, St. Paul, MN, pp. 1101–1108 (2013)

    Google Scholar 

  10. Zhang, K., Maeda, Y., Takahashi, Y.: Group behavior learning in multi-agent systems based on social interaction among agents. SCIS & ISIS 12010, 193–198 (2010)

    Google Scholar 

  11. Åström, K.: Optimal control of markov processes with incomplete state information. Journal of Mathematical Analysis and Applications 10(1), 174–205 (1965)

    MathSciNet  CrossRef  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J. Alejandro Camargo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Camargo, J.A., Barrios-Aranibar, D. (2016). \(\delta \)-Radius Unified Influence Value Reinforcement Learning. In: , et al. Distributed Computing and Artificial Intelligence, 13th International Conference. Advances in Intelligent Systems and Computing, vol 474. Springer, Cham. https://doi.org/10.1007/978-3-319-40162-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-40162-1_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40161-4

  • Online ISBN: 978-3-319-40162-1

  • eBook Packages: EngineeringEngineering (R0)