Advertisement

Autonomous Agents and Multi-Agent Systems

, Volume 15, Issue 1, pp 47–90 | Cite as

Learning to communicate in a decentralized environment

  • Claudia V. Goldman
  • Martin Allen
  • Shlomo Zilberstein
Article

Abstract

Learning to communicate is an emerging challenge in AI research. It is known that agents interacting in decentralized, stochastic environments can benefit from exchanging information. Multi-agent planning generally assumes that agents share a common means of communication; however, in building robust distributed systems it is important to address potential miscoordination resulting from misinterpretation of messages exchanged. This paper lays foundations for studying this problem, examining its properties analytically and empirically in a decision-theoretic context. We establish a formal framework for the problem, and identify a collection of necessary and sufficient properties for decision problems that allow agents to employ probabilistic updating schemes in order to learn how to interpret what others are communicating. Solving the problem optimally is often intractable, but our approach enables agents using different languages to converge upon coordination over time. Our experimental work establishes how these methods perform when applied to problems of varying complexity.

Keywords

Multi-agent systems Learning Communication Probabilistic methods 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allen, M., Goldman, C. V., & Zilberstein, S. (2005). Learning to communicate in decentralized systems. In Proceedings of the workshop on multiagent learning, twentieth national conference on artificial intelligence (pp. 1–8). (AAAI Tech. Report WS-05-09). PA: Pittsburgh.Google Scholar
  2. 2.
    Balch T. and Arkin R.C. (1994). Communication in reactive multiagent robotic systems. Autonomous Robots 1: 1–25CrossRefGoogle Scholar
  3. 3.
    Becker R., Zilberstein S., Lesser V. and Goldman C.V. (2004). Solving transition independent decentralized MDPs. Journal of Artificial Intelligence Research 22: 423–455zbMATHMathSciNetGoogle Scholar
  4. 4.
    Bernstein D.S., Givan R., Immerman N. and Zilberstein S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research 27(4): 819–840zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Bhattacharjee Y. (2004). From heofonum to heavens. Science 303: 1326–1328CrossRefGoogle Scholar
  6. 6.
    Blume A. and Sobel J. (1995). Communication-proof equilibria in cheap-talk games. Journal of Economic Theory 65: 359–382zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Bonarini A. and Sassaroli P. (1997). Opportunistic multimodel diagnosis with imperfect models. Information Sciences 103(1–4): 161–185CrossRefGoogle Scholar
  8. 8.
    Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the sixteenth international joint conference on artificial intelligence (pp. 478–485). Sweden: Stockholm.Google Scholar
  9. 9.
    Coradeschi S. and Saffiotti A. (2003). An introduction to the anchoring problem. Robotics and Autonomous Systems 43(2–3): 85–96CrossRefGoogle Scholar
  10. 10.
    Davidson D. (1984). Inquiries into Truth and Interpretation. Oxford University Press, Oxford, EnglandGoogle Scholar
  11. 11.
    Decker K.S. and Lesser V.R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent Cooperative Information Systems 1(2): 319–346CrossRefGoogle Scholar
  12. 12.
    Doucet, A. (1998). On sequential simulation-based methods for bayesian filtering. Technical Report CUED/F-INFENG/TR.310 Department of Engineering, University of Cambridge.Google Scholar
  13. 13.
    Durfee E.H. (1988). Coordination of Distributed Problem Solvers. Kluwer Academic, Boston, MAzbMATHGoogle Scholar
  14. 14.
    Durfee E.H. and Lesser V.R. (1988). Using partial global plans to coordinate distributed problem solvers. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence, pp 285–293. Morgan Kaufmann Publishers Inc., San Mateo, CAGoogle Scholar
  15. 15.
    Finin T., Labrou Y. and Mayfield J. (1997). KQML as an agent communication language. In: Bradshaw, J. (eds) Software Agents, pp. MIT Press, Cambridge, MAGoogle Scholar
  16. 16.
    Firoiu, L., Oates, T., & Cohen, P. (1998). Learning regular languages from positive evidence. In Proceedings of the twentieth annual meeting of the cognitive science society (pp. 350–355). WI: Madison.Google Scholar
  17. 17.
    Gmytrasiewicz P. and Doshi P. (2005). A framework for sequential planning in multiagent settings. Journal of AI Research 24: 1–31Google Scholar
  18. 18.
    Gmytrasiewicz, P. J., Summers, M., & Gopal, D. (2002). Toward automated evolution of agent communication languages. In Proceedings of the thirtyfifth hawaii international conference on system sciences, Hawaii, p. 79.Google Scholar
  19. 19.
    Goldman, C. V., Allen, M., & Zilberstein, S. (2004). Decentralized language learning through acting. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 1006–1013). New York, NY, 2004.Google Scholar
  20. 20.
    Goldman, C. V., Allen, M., & Zilberstein, S. (2006). Learning to communicate in a decentralized environment. Technical Report UM-CS-2006-16, 2006. Department of Computer Science, University of Massachusetts.Google Scholar
  21. 21.
    Goldman, C. V., & Rosenschein, J. S. (1994). Emergent coordination through the use of cooperative state-changing rules. In Proceedings of the twelfth national conference on artificial intelligence (pp. 408–413). WA: Seattle.Google Scholar
  22. 22.
    Goldman, C. V., & Zilberstein, S. (2003). Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the second international joint conference on autonomous agents and multi-agent systems (pp. 137—144). Melbourne, Australia.Google Scholar
  23. 23.
    Goldman C.V. and Zilberstein S. (2004). Decentralized control of cooperative systems: Categorization and complexity analysis. Journal of Artificial Intelligence Research 22: 143–174zbMATHMathSciNetGoogle Scholar
  24. 24.
    Goldman, C. V., & Zilberstein, S. (2004). Goal-oriented Dec-MDPs with direct communication. Technical Report UM-CS-2004-44, Department of Computer Science, University of Massachusetts.Google Scholar
  25. 25.
    Grosz B.J. and Kraus S. (1996). Collaborative plans for complex group action. Artificial Intelligence 86(2): 269–357CrossRefMathSciNetGoogle Scholar
  26. 26.
    Kaelbling L.P. (1993). Learning in Embedded Systems. MIT Press, Cambridge, MAGoogle Scholar
  27. 27.
    Komarova N. and Niyogi P. (2004). Optimizing the mutual intelligibility of linguistic agents in a shared world. Artificial Intelligence 154: 1–42zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    MacLennan, B. (1990). Evolution of communication in a population of simple machines. Knoxville, Technical Report CS90-99, Department of Computer Science, University of Tennessee.Google Scholar
  29. 29.
    NASA (1999). Mars climate orbiter failure board report. Available at: ftp://ftp.hq.nasa.gov/pub/ pao/reports/1999/MCO_report.pdfGoogle Scholar
  30. 30.
    Puterman M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, NYzbMATHGoogle Scholar
  31. 31.
    Putnam, H. (1975). The meaning of ‘meaning’. In Mind, Language and Reality: Philosophical Papers (pp. 215–271). Vol. 2 Cambridge, England: Cambridge University Press.Google Scholar
  32. 32.
    Quine W.V.O. (1960). Word and Object. MIT Press, Cambridge, MAzbMATHGoogle Scholar
  33. 33.
    Serafini L. and Bouquet P. (2004). Comparing formal theories of context in AI. Artificial Intelligence 155: 41–67zbMATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Sharygina N. and Peled D. (2001). A combined testing and verification approach for software reliability. Formal Methods for Increasing Software Productivity, LNCS 2021: 611–628Google Scholar
  35. 35.
    Smith R.G. (1988). The contract net protocol: High level communication and control in a distributed problem solver. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence., pp 357–366. Morgan Kaufmann Publishers Inc., San Mateo, CaliforniaGoogle Scholar
  36. 36.
    Steels, L., & Vogt, P. (1997). Grounding adaptive language games in robotic agents. In Proceedings of the fourth european conference on artificial life (pp. 474–482). UK: Brighton.Google Scholar
  37. 37.
    Sutton R.S. and Barto A.G. (2000). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MAGoogle Scholar
  38. 38.
    Thrun S., Fox D., Burgard W. and Dellaert F. (2001). Robust Monte Carlo localization for mobile robots. Artificial Intelligence 128: 99–141zbMATHCrossRefGoogle Scholar
  39. 39.
    Vogt P., Coumans H. (2003). Investigating social interaction strategies for bootstrapping lexicon development, Journal of Artificial Societies and Social Simulation, 6(1). http://jasss.soc.surrey.ac.uk/6/1/4.html
  40. 40.
    Wang, J., & Gasser, L. (2002). Mutual online concept learning for multiple agents. In Proceedings of the first international joint conference on autonomous agents and multi-agent systems (pp. 362–369). Bologna, Italy.Google Scholar
  41. 41.
    Wärneryd K. (1993). Cheap talk, coordination and evolutionary stability. Games and Economic Behavior 5: 532–546zbMATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Yanco, H. A. (1994). Robot communication: Issues and implementation. Master’s thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.Google Scholar
  43. 43.
    Yanco H.A. and Stein L.A. (1993). An adaptive communication protocol for cooperating mobile robots. In: Meyer, J.A., Roitblat, H.L. and Wilson, S.W. (eds) From Animals to Animats: Proceedings of the second international conference on the simulation of adaptive behavior, pp 478–485. MIT Press, Cambridge, MAGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2006

Authors and Affiliations

  • Claudia V. Goldman
    • 1
  • Martin Allen
    • 2
  • Shlomo Zilberstein
    • 2
  1. 1.Caesarea Rothschild InstituteUniversity of HaifaHaifaIsrael
  2. 2.Department of Computer ScienceUniversity of Massachusetts AmherstAmherstUSA

Personalised recommendations