Autonomous Agents and Multi-Agent Systems

, Volume 11, Issue 3, pp 387–434 | Cite as

Cooperative Multi-Agent Learning: The State of the Art

Article

Abstract

Cooperative multi-agent systems (MAS) are ones in which several agents attempt, through their interaction, to jointly solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task of programming solutions to MAS problems has spawned increasing interest in machine learning techniques to automate the search and optimization process. We provide a broad survey of the cooperative multi-agent learning literature. Previous surveys of this area have largely focused on issues common to specific subareas (for example, reinforcement learning, RL or robotics). In this survey we attempt to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics. We find that this broad view leads to a division of the work into two categories, each with its own special issues: applying a single learner to discover joint solutions to multi-agent problems (team learning), or using multiple simultaneous learners, often one per agent (concurrent learning). Additionally, we discuss direct and indirect communication in connection with learning, plus open issues in task decomposition, scalability, and adaptive dynamics. We conclude with a presentation of multi-agent learning problem domains, and a list of multi-agent learning resources.

Key words

multi-agent systems machine learning multi-agent learning cooperation survey 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D. H. Ackley and M. Littman, “Altruism in the evolution of communication,” in Artificial Life IV: Proceedings of the International Workshop on the Synthesis and Simulation of Living Systems, (3rd edn.), MIT Press, 1994.Google Scholar
  2. 2.
    D. Andre, F. Bennett III, and J. Koza, “Discovery by genetic programming of a cellular automata rule that is better than any known rule for the majority classification problem,” in Genetic Programming 1996: Proceedings of the First Annual Conference, MIT Press, 1996.Google Scholar
  3. 3.
    D. Andre and A. Teller, “Evolving team Darwin United,” in M. Asada and H. Kitano, (eds.), RoboCup-98: Robot Soccer World Cup II, Springer Verlag, 1999.Google Scholar
  4. 4.
    P. Angeline and J. Pollack, “Competitive environments evolve better solutions for complex tasks,” in S. Forrest, (ed.), Proceedings of the Fifth International Conference on Genetic Algorithms (ICGA), Morgan Kaufmann: San Mateo, CA, pp. 264–270, 1993.Google Scholar
  5. 5.
    Arthur, W. 1994“Inductive reasoning and bounded rationality”Complex. Econ. Theory84406411Google Scholar
  6. 6.
    T. Bäck, Evolutionary Algorithms in Theory and Practice: Evolutionary Straegies, Evolutionary Programming, and Genetic Algorithms, Oxford Press, 1996.Google Scholar
  7. 7.
    T. Balch, Learning roles: Behavioral diversity in robot teams, Technical Report GIT-CC-97-12, Georgia Institute of Technology, 1997.Google Scholar
  8. 8.
    T. Balch, Behavioral Diversity in Learning Robot Teams, PhD thesis, College of Computing, Georgia Institute of Technology, 1998.Google Scholar
  9. 9.
    T. Balch, “Reward and diversity in multirobot foraging,” in IJCAI-99 Workshop on Agents Learning About, From and With other Agents, 1999.Google Scholar
  10. 10.
    B. Banerjee, R. Mukherjee, and S. Sen. “Learning mutual trust,” in Working Notes of AGENTS-00 Workshop on Deception, Fraud and Trust in Agent Societies, pp. 9–14, 2000.Google Scholar
  11. 11.
    A. Barto, R. Sutton, and C. Watkins, “Learning and sequential decision making,” in M. Gabriel and J. Moore, (eds.), Learning and Computational Neuroscience: Foundations of Adaptive Networks, MIT Press: Cambridge, MA, 1990.Google Scholar
  12. 12.
    J. Bassett and K. De Jong, “Evolving behaviors for cooperating agents,” in Z. Ras, (ed.), Proceedings from the Twelfth International Symposium on Methodologies for Intelligent Systems, Springer-Verlag: Charlotte, NC, pp. 157–165, 2000.Google Scholar
  13. 13.
    Bassett, J. K. 2002A study of generalization techniques in evolutionary rule learningGeorge Mason UniversityFairfax VA, USAMaster’s thesisGoogle Scholar
  14. 14.
    R. Beckers, O. E. Holland, and J.-L. Deneubourg. “From local actions to global tasks: Stigmergy and collective robotics,” in Artificial Life IV: Proceedings of the International Workshop on the Synthesis and Simulation of Living Systems, (3rd edn.), MIT Press, 1994.Google Scholar
  15. 15.
    M. Benda, V. Jagannathan, and R. Dodhiawala, On optimal cooperation of knowledge sources - an empirical investigation, Technical Report BCS-G2010-28, Boeing Advanced Technology Center, Boeing Computer Services, 1986.Google Scholar
  16. 16.
    H. Berenji and D. Vengerov, “Advantages of cooperation between reinforcement learning agents in difficult stochastic problems,” in Proceedings of 9th IEEE International Conference on Fuzzy Systems, 2000.Google Scholar
  17. 17.
    H. Berenji and D. Vengerov, Learning, cooperation, and coordination in multi-agent systems, Technical Report IIS-00-10, Intelligent Inference Systems Corp., 333 W. Maude Avennue, Suite 107, Sunnyvale, CA 94085-4367, 2000.Google Scholar
  18. 18.
    D. Bernstein, S. Zilberstein, and N. Immerman, “The complexity of decentralized control of MDPs,” in Proceedings of UAI-2000: The Sixteenth Conference on Uncertainty in Artificial Intelligence, 2000.Google Scholar
  19. 19.
    H. J. Blumenthal and G. Parker, “Co-evolving team capture strategies for dissimilar robots,” in’Proceedings of Artificial Multiagent Learning. Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004.Google Scholar
  20. 20.
    E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, SFI Studies in the Sciences of Complexity, Oxford University Press, 1999.Google Scholar
  21. 21.
    J. C. Bongard, “The legion system: A novel approach to evolving heterogeneity for collective problem solving” in R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty, (eds.), Genetic Programming: Proceedings of EuroGP-2000. Vol. 1802, Edinburgh, 15–16 2000. Springer-Verlag. ISBN 3-540-67339-3, pp. 16–28.Google Scholar
  22. 22.
    C. Boutilier, “Learning conventions in multiagent stochastic domains using likelihood estimates,” in’Uncertainty in Artificial Intelligence, pp. 106–114, 1996.Google Scholar
  23. 23.
    C. Boutilier, “Planning, learning and coordination in multiagent decision processes,” in Proceedings of the Sixth Conference on Theoretical Aspects of Rationality and Knowledge (TARK96), pp. 195–210, 1996.Google Scholar
  24. 24.
    M. Bowling, “Convergence problems of general-sum multiagent reinforcement learning,” in’Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 89–94, 2000.Google Scholar
  25. 25.
    M. Bowling, Multiagent Learning in the Presence of Agents with Limitations, PhD thesis, Computer Science Department, Carnegie Mellon University, 2003.Google Scholar
  26. 26.
    M. Bowling and M. Veloso, An analysis of stochastic game theory for multiagent reinforcement learning, Technical Report CMU-CS-00–165, Computer Science Department, Carnegie Mellon University, 2000.Google Scholar
  27. 27.
    M. Bowling and M. Veloso, “Rational and convergent learning in stochastic games,” in Proceedings of Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-01), pp. 1021–1026, 2001.Google Scholar
  28. 28.
    M. Bowling and M. Veloso, Existence of multiagent equilibria with limited agents, Technical Report CMU-CS-02-104, Computer Science Department, Carnegie Mellon University, 2002.Google Scholar
  29. 29.
    Bowling, M., Veloso, M. 2002“Multiagent learning using a variable learning rate”Artif. Intell136215250CrossRefGoogle Scholar
  30. 30.
    J. A. Boyan and M. Littman, “Packet routing in dynamically changing networks: A reinforcement learning approach,” in J. D. Cowan, G. Tesauro, and J. Alspector, (eds.), Advances in Neural Information Processing Systems, Vol. 6, Morgan Kaufmann, pp. 671–678, 1994.Google Scholar
  31. 31.
    R. Brafman and M. Tennenholtz, “Efficient learning equilibrium,” in Advances in Neural Information Processing Systems (NIPS-2002), 2002.Google Scholar
  32. 32.
    W. Brauer and G. Weiß, “Multi-machine scheduling - a multi-agent learning approach,” in Proceedings of the Third International Conference on Multi-Agent Systems, pp. 42–48, 1998.Google Scholar
  33. 33.
    P. Brazdil, M. Gams, S. Sian, L. Torgo, and W. van de Velde, “Learning in distributed systems and multi-agent environments,” in Y. Kodratoff, (ed.), Lecture Notes in Artificial Intelligence, Vol. 482, Springer-Verlag, pp. 412–423, 1991.Google Scholar
  34. 34.
    Buffet, O., Dutech, A., Charpillet, F. 2001

    “Incremental reinforcement learning for designing multi-agent systems”

    Müller, J.P.Andre, E.Sen, S.Frasson, C. eds. Proceedings of the Fifth International Conference on Autonomous AgentsACM PressMontreal, Canada3132
    Google Scholar
  35. 35.
    O. Buffet, A. Dutech, and F. Charpillet, “Learning to weigh basic behaviors in scalable agents,” in Proceedings of the 1st International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS’02), 2002.Google Scholar
  36. 36.
    H. Bui, S. Venkatesh, and D. Kieronska, “A framework for coordination and learning among team of agents,” in W. Wobcke, M. Pagnucco, and C. Zhang, (eds.), Agents and Multi-Agent Systems: Formalisms, Methodologies and Applications, Lecture Notes in Artificial Intelligence. Vol. 1441, Springer-Verlag, pp. 164–178, 1998.Google Scholar
  37. 37.
    Bui, H., Venkatesh, S., Kieronska, D. 1999“Learning other agents’ preferences in multi-agent negotiation using the Bayesian classifier”Int. J. Coop. Inform. Syst.8275294CrossRefGoogle Scholar
  38. 38.
    L. Bull, “Evolutionary computing in multi-agent environments: Partners,” in T. Back, (ed.), Proceedings of the Seventh International Conference on Genetic Algorithms, Morgan Kaufmann, pp. 370–377, 1997.Google Scholar
  39. 39.
    L. Bull, “Evolutionary computing in multi-agent environments: Operators,” in D. W. V. W. Porto, N. Saravanan, and A. E. Eiben, (eds.), Proceedings of the Seventh Annual Conference on Evolutionary Programming, Springer Verlag, pp. 43–52, 1998.Google Scholar
  40. 40.
    L. Bull and T. C. Fogarty, “Evolving cooperative communicating classifier systems”, in A. V. Sebald and L. J. Fogel, (eds.), Proceedings of the Fourth Annual Conference on Evolutionary Programming (EP94), pp. 308–315, 1994.Google Scholar
  41. 41.
    L. Bull and O. Holland, “Evolutionary computing in multiagent environments: Eusociality”, in Proceedings of Seventh Annual Conference on Genetic Algorithms, 1997.Google Scholar
  42. 42.
    Cangelosi, A. 2001“Evolution of communication and language using signals, symbols, and words”IEEE Trans. Evol. Comput.593101CrossRefGoogle Scholar
  43. 43.
    Cao, Y.U., Fukunaga, A.S., Kahng, A.B. 1997“Cooperative mobile robotics: Antecedents and directions”Auton. Robots4723CrossRefGoogle Scholar
  44. 44.
    D. Carmel, Model-based Learning of Interaction Strategies in Multi-agent systems, PhD thesis, Technion - Israel Institute of Technology, 1997.Google Scholar
  45. 45.
    D. Carmel and S. Markovitch, The M* algorithm: Incorporating opponent models into adversary search. Technical Report 9402, Technion - Israel Institute of Technology, March 1994Google Scholar
  46. 46.
    L.-E. Cederman, Emergent Actors in World Politics: How States and Nations Develop and Dissolve, Princeton University Press, 1997Google Scholar
  47. 47.
    G. Chalkiadakis and C. Boutilier, “Coordination in multiagent reinforcement learning: A Bayesian approach,” in Proceedings of The Second International Joint Conference on Autonomous Agents & Multiagent Systems (AAMAS 2003). ACM, 2003. ISBN 1-58113-683-8Google Scholar
  48. 48.
    H. Chalupsky, Y. Gil, C. A. Knoblock, K. Lerman, J. Oh, D. Pynadath, T. Russ, and M. Tambe, “Electric elves: Agent technology for supporting human organizations,” in AI Magazine - Summer 2002, AAAI Press, 2002Google Scholar
  49. 49.
    Y.-H. Chang, T. Ho, and L. Kaelbling, “All learning is local: Multi-agent learning in global reward games,” in Proceedings of Neural Information Processing Systems (NIPS-03), 2003Google Scholar
  50. 50.
    Y.-H. Chang, T. Ho, and L. Kaelbling, “Multi-agent learning in mobilized ad-hoc networks,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium, Technical Report FS-04-02, 2004Google Scholar
  51. 51.
    C. Claus and C. Boutilier, “The dynamics of reinforcement learning in cooperative multiagent systems,” in Proceedings of National Conference on Artificial Intelligence AAAI/IAAI, pp. 746–752, 1998Google Scholar
  52. 52.
    D. Cliff and G. F. Miller, “Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations”, in Proceedings of the Third European Conference on Artificial Life, Springer-Verlag, pp. 200–218, 1995Google Scholar
  53. 53.
    Collins, R., Jefferson, D. 1991

    “An artificial neural network representation for artificial organisms”

    Schwefel, H.-P.M´′anner, R. eds. Parallel Problem Solving from Nature: 1st Workshop (PPSN I)Springer-VerlagBerlin259263
    Google Scholar
  54. 54.
    Collins, R., Jefferson, D. 1992

    “AntFarm: Towards simulated evolution”

    Langton, C.Taylor, C.Farmer, J.D.Rasmussen, S. eds. Artificial Life IIAddison-WesleyRedwood City, CA579601
    Google Scholar
  55. 55.
    E. Crawford and M. Veloso, “Opportunities for learning in multi-agent meeting scheduling”, in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004Google Scholar
  56. 56.
    V. Crespi, G. Cybenko, M. Santini, and D. Rus. Decentralized control for coordinated flow of multi-agent systems. Technical Report TR2002-414, Dartmouth College, Computer Science, Hanover, NH, January 2002Google Scholar
  57. 57.
    R. H. Crites, Large-Scale Dynamic Optimization Using Teams of Reinforcement Learning Agents, PhD thesis, University of Massachusetts Amherst, 1996Google Scholar
  58. 58.
    Cutkosky, M.R., Englemore, R.S., Fikes, R.E., Genesereth, M.R., Gruber, T.R., Mark, W.S., Tenenbaum, J.M., Weber, J.C. 1997

    “PACT: An experiment in integrating concurrent engineering systems”

    Huhns, M.N.Singh, M.P. eds. Readings in AgentsMorgan KaufmannSan Francisco, CA, USA4655
    Google Scholar
  59. 59.
    T. Dahl, M. Mataric, and G. Sukhatme, “Adaptive spatio-temporal organization in groups of robots,” in Proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-02), 2002Google Scholar
  60. 60.
    R. Das, M. Mitchell, and J. Crutchfield, “A genetic algorithm discovers particle-based computation in cellular automata”, in Parallel Problem Solving from Nature III, LNCS 866, Springer-Verlag, pp.’344–353, 1994Google Scholar
  61. 61.
    J. Davis and G. Kendall, “An investigation, using co-evolution, to evolve an awari player,” in Proceedings of 2002 Congress on Evolutionary Computation (CEC2002), 2002Google Scholar
  62. 62.
    B. de Boer, “Generating vowel systems in a population of agents,” in Proceedings of the Fourth European Conference Artificial Life, MIT Press, 1997Google Scholar
  63. 63.
    Jong, K. 1975An Analysis of the Behavior of a Class of Genetic Adaptive SystemsUniversity of MichiganAnn Arbor, MIPhD thesisGoogle Scholar
  64. 64.
    K. De Jong, Evolutionary Computation: A Unified Approach, MIT Press, 2005Google Scholar
  65. 65.
    K. Decker, E. Durfee, and V. Lesser, “Evaluating research in cooperative distributed problem solving,” in L. Gasser and M. Huhns, (eds.), Distributed Artificial Intelligence Volume II, Pitman Publishing and Morgan Kaufmann, pp. 487–519, 1989Google Scholar
  66. 66.
    Decker, K. Fisher, M. Luck, M. Tennenholtz, M. UKMAS’98 Contributors1999“Continuing research in multi-agent systems”Knowl. Eng. Rev.14279283CrossRefGoogle Scholar
  67. 67.
    J. L. Deneubourg, S. Goss, N. Franks, A. Sendova-Franks, C. Detrain, and L. Chretien, “The dynamics of collective sorting: Robot-like ants and ant-like robots,” in From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, MIT Press, pp.’356–363, 1991Google Scholar
  68. 68.
    J. Denzinger and M. Fuchs, “Experiments in learning prototypical situations for variants of the pursuit game,” in Proceedings on the International Conference on Multi-Agent Systems (ICMAS-1996), pp. 48–55, 1996Google Scholar
  69. 69.
    M. Dowell. Learning in Multiagent Systems, PhD thesis, University of South Carolina, 1995Google Scholar
  70. 70.
    K. Dresner and P. Stone, “Multiagent traffic management: A reservation-based intersection control mechanism,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  71. 71.
    G. Dudek, M. Jenkin, R. Milios, and D. Wilkes, “A taxonomy for swarm robots,” in Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems, 1993Google Scholar
  72. 72.
    E. Durfee, “What your computer really needs to know, you learned in kindergarten,” in National Conference on Artificial Intelligence, pp. 858–864, 1992Google Scholar
  73. 73.
    Durfee, E., Lesser, V., Corkill, D. 1987“Coherent cooperation among communicating problem solvers”IEEE Trans. ComputC-3612751291Google Scholar
  74. 74.
    Durfee, E., Lesser, V., Corkill, D. March 1989Trends in cooperative distributed problem solvingIEEE Trans. Knowl. Data Eng.KDE-16383CrossRefGoogle Scholar
  75. 75.
    A. Dutech, O. Buffet, and F. Charpillet, “Multi-agent systems by incremental gradient reinforcement learning,” in Proceedings of Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-01), pp. 833–838, 2001Google Scholar
  76. 76.
    Fernandez, F., Parker, L. 2001“Learning in large cooperative multi-robot domains,”Int. J. Robot. Autom.16217226Google Scholar
  77. 77.
    Ficici, S., Pollack, J.,  et al. 1998

    “Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states”

    Adami, C. eds. Proceedings of the Sixth International Conference on Artificial LifeMIT PressCambridge, MA238247
    Google Scholar
  78. 78.
    S. Ficici and J. Pollack, “A game-theoretic approach to the simple coevolutionary algorithm”, in Proceedings of the Sixth International Conference on Parallel Problem Solving from Nature (PPSN VI). Springer Verlag, 2000Google Scholar
  79. 79.
    K. Fischer, N. Kuhn, H. J. Muller, J. P. Muller, and M. Pischel, “Sophisticated and distributed: The transportation domain,” in Proceedings of the Fifth European Workshop on Modelling Autonomous Agents in a Multi-Agent World (MAAMAW’93), 1993Google Scholar
  80. 80.
    D. Fogel, Blondie24: Playing at the Edge of Artificial Intelligence, Morgan Kaufmann, 2001. ISBN 1-55860-783-8Google Scholar
  81. 81.
    L. Fogel, Intelligence Through Simulated Evolution: Forty Years of Evolutionary Programming, Wiley Series on Intelligent Systems, 1999Google Scholar
  82. 82.
    D. Fudenberg and D. Levine, The Theory of Learning in Games, MIT Press, 1998Google Scholar
  83. 83.
    Garland, A., Alterman, R. 2004“Autonomous agents that learn to better coordinate”Auton. Agents Multi-Agent Syst.8267301CrossRefGoogle Scholar
  84. 84.
    M. Ghavamzadeh and S. Mahadevan, “Learning to communicate and act using hierarchical reinforcement learning,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  85. 85.
    Glance, N., Huberman, B. 1994“The dynamics of social dilemmas”Sci. Am.2707681MarchGoogle Scholar
  86. 86.
    P. Gmytrasiewicz, A Decision-Theoretic Model of Coordination and Communication in Autonomous Systems (Reasoning Systems), PhD thesis, University of Michigan, 1992Google Scholar
  87. 87.
    Goldberg, D. E. 1989Genetic Algorithms in Search, Optimization, and Machine LearningAddison WesleyReading, MAGoogle Scholar
  88. 88.
    Goldman, C., Rosenschein, J. 1996

    “Mutually supervised learning in multiagent systems”

    Weiß, G.Sen, S. eds. Adaptation and Learning in Multi-Agent SystemsSpringer-VerlagHeidelberg, Germany. Berlin8596
    Google Scholar
  89. 89.
    B. M. Good, Evolving multi-agent systems: Comparing existing approaches and suggesting new directions, Master’s thesis, University of Sussex, 2000Google Scholar
  90. 90.
    M. Gordin, S. Sen, and N. Puppala, “Evolving cooperative groups: Preliminary results”, in Working Papers of the AAAI-97 Workshop on Multiagent Learning, pp. 31–35, 1997Google Scholar
  91. 91.
    Grand, S., Cliff, D. 1998“Creatures: Entertainment software agents with artificial life”Auton. Agents Multi-Agent Syst.13957CrossRefGoogle Scholar
  92. 92.
    S. Grand, D. Cliff, and A. Malhotra, “Creatures : Artificial life autonomous software agents for home entertainment”, in Proceedings of the First International Conference on Autonomous Agents (Agents-97), pp. 22–29, 1997Google Scholar
  93. 93.
    D. L. Grecu, Using Learning to Improve Multi-Agent Systems for Design. PhD thesis, Worcester Polytechnic Institute, 1997Google Scholar
  94. 94.
    A. Greenwald, J. Farago, and K. Hall, “Fair and efficient solutions to the Santa Fe bar problem,” in’Proceedings of the Grace Hopper Celebration of Women in Computing 2002, 2002Google Scholar
  95. 95.
    A. Greenwald and K. Hall, “Correlated Q-learning,” in Proceedings of the Twentieth International Conference on Machine Learning, 2003Google Scholar
  96. 96.
    Grefenstette, J. 1991

    “Lamarckian learning in multi-agent environments”

    Belew, R.Booker, L. eds. Proceedings of the Fourth International Conference on Genetic AlgorithmsMorgan KaufmanSan Mateo, CA303310
    Google Scholar
  97. 97.
    Grefenstette, J., Ramsey, C.L., Schultz, A. 1990“Learning sequential decision rules using simulation models and competition”Machine Learn.5355381Google Scholar
  98. 98.
    C. Guestrin, M. Lagoudakis, and R. Parr, “Coordinated reinforcement learning,” in Proceedings of the 2002 AAAI Symposium Series: Collaborative Learning Agents, 2002Google Scholar
  99. 99.
    Gustafson, S.M. 2000Layered learning in genetic programming for a co-operative robot soccer problemKansas State UniversityManhattan, KS, USAMaster’s thesisGoogle Scholar
  100. 100.
    Gustafson, S.M., Hsu, W. H. 2001

    “Layered learning in genetic programming for a co-operative robot soccer problem,”

    Miller, J. F.Tomassini, M.Lanzi, P. L.Ryan, C.Tettamanzi, A. G. B.Langdon, W. B. eds. Genetic Programming: Proceedings of EuroGP-2001Springer-VerlagLake Como, Italy29130118–20 ISBN 3-540-41899-7
    Google Scholar
  101. 101.
    A. Hara and T. Nagao, “Emergence of cooperative behavior using ADG; Automatically Defined Groups,” in Proceedings of the 1999 Genetic and Evolutionary Computation Conference (GECCO-99), pp. 1038–1046, 1999Google Scholar
  102. 102.
    I. Harvey, P. Husbands, D. Cliff, A. Thompson, and N. Jakobi, “Evolutionary robotics: The Sussex approach,” Robot. Auton. Syst., 1996Google Scholar
  103. 103.
    T. Haynes, K. Lau, and S. Sen, “Learning cases to compliment rules for conflict resolution in multiagent systems,” in S. Sen, (ed.), AAAI Spring Symposium on Adaptation, Coevolution, and Learning in Multiagent Systems, pp. 51–56, 1996Google Scholar
  104. 104.
    T., Haynes, Sen, S. 1995

    “Evolving behavioral strategies in predators and prey”

    Weiß, G.Sen, S. eds. Adaptation and Learning in Multiagent Systems, Lecture Notes in Artificial IntelligenceSpringer VerlagBerlin, Germany
    Google Scholar
  105. 105.
    T. Haynes and S. Sen, “Adaptation using cases in cooperative groups,” in I. Imam (ed.), Working Notes of the AAAI-96 Workshop on Intelligent Adaptive Agents, Portland, OR, 1996Google Scholar
  106. 106.
    T. Haynes and S. Sen, Cooperation of the fittest, Technical Report UTULSA-MCS-96-09, The University of Tulsa, Apr. 12, 1996Google Scholar
  107. 107.
    T. Haynes and S. Sen, “Learning cases to resolve conflicts and improve group behavior,” in M. Tambe and P. Gmytrasiewicz, (eds.), Working Notes of the AAAI-96 Workshop on Agent Modeling, Portland, OR, pp. 46–52, 1996Google Scholar
  108. 108.
    T. Haynes and S. Sen, “Crossover operators for evolving a team,” in J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo, (eds.), Genetic Programming 1997: Proceedings of the Second Annual Conference, Morgan Kaufmann: Stanford University, CA, USA, pp. 162–167, 13–16 July 1997Google Scholar
  109. 109.
    T. Haynes, S. Sen, D. Schoenefeld, and R. Wainwright, “Evolving a team,” in E. V. Siegel and J. R. Koza, (eds.), Working Notes for the AAAI Symposium on Genetic Programming, AAAI: MIT, Cambridge, MA, USA, pp. 23–30, 10–12 Nov. 1995Google Scholar
  110. 110.
    T. Haynes, S. Sen, D. Schoenefeld, and R. Wainwright, Evolving multiagent coordination strategies with genetic programming, Technical Report UTULSA-MCS-95–04, The University of Tulsa, May 31, 1995Google Scholar
  111. 111.
    T. Haynes, R. Wainwright, S. Sen, and D. Schoenefeld, “Strongly typed genetic programming in evolving cooperation strategies,” in L. Eshelman, (ed.), Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), Morgan Kaufmann: Pittsburgh, PA, USA, pp. 271–278, ISBN 1-55860-370-0, 15-19 July 1995Google Scholar
  112. 112.
    T. D. Haynes and S. Sen, “Co-adaptation in a team,” Int. J. Comput. Intell. Org. (IJCIO), 1997Google Scholar
  113. 113.
    Hillis, D. 1991“Co-evolving parasites improve simulated evolution as an optimization procedure,”Artif. Life II, SFI Stud. Sci. Complex.10313324Google Scholar
  114. 114.
    Holland, J. 1975Adaptation in Natural and Artificial SystemsThe MIT PressCambridge, MAGoogle Scholar
  115. 115.
    J. Holland, “Properties of the bucket brigade,” in Proceedings of an International Conference on Genetic Algorithms, 1985Google Scholar
  116. 116.
    B. Hölldobler and E. O. Wilson, The Ants, Harvard University Press, 1990Google Scholar
  117. 117.
    W. H. Hsu and S. M. Gustafson, “Genetic programming and multi-agent layered learning by reinforcements,” in W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K.’Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska, (eds.), GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers: New York, 9-13 July 2002, ISBN 1-55860-878-8, pp. 764-771Google Scholar
  118. 118.
    J. Hu and M. Wellman, “Self-fulfilling bias in multiagent learning,” in Proceedings of the Second International Conference on Multi-Agent Systems, 1996Google Scholar
  119. 119.
    J. Hu and M. Wellman, “Multiagent reinforcement learning: Theoretical framework and an algorithm,” in Proceedings of the Fifteenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 242–250, 1998Google Scholar
  120. 120.
    J. Hu and M. Wellman, “Online learning about other agents in a dynamic multiagent system,” in K.’P. Sycara and M. Wooldridge, (eds.), Proceedings of the Second International Conference on Autonomous Agents (Agents’98), ACM Press: New York, 1998, pp. 239–246, ISBN 0-89791-983-1Google Scholar
  121. 121.
    Hu, J., Wellman, M. 2003“Nash Q-learning for general-sum stochastic games”J. Machine Learn. Res.410391069CrossRefGoogle Scholar
  122. 122.
    Huang, J., Jennings, N. R., Fox, J. 1995

    “An agent architecture for distributed medical care,”

    Wooldridge, M.Jennings, N. R. eds. Intelligent Agents: Theories, Architectures, and Languages (LNAI Volume 890)Springer-VerlagHeidelberg, Germany219232
    Google Scholar
  123. 123.
    M. Huhns and M. Singh, “Agents and multiagent systems: Themes, approaches and challenges,” in’M. Huhns and M. Singh, (eds.), Readings in Agents, Morgan Kaufmann, pp. 1–23, 1998Google Scholar
  124. 124.
    M. Huhns and G. Weiß, “Special issue on multiagent learning,” Machine Learn. J., vol. 33, nos. 2–3, 1998Google Scholar
  125. 125.
    H. Iba, “Emergent cooperation for multiple agents using genetic programming,” in H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, (eds.), Parallel Problem Solving from Nature IV: Proceedings of the International Conference on Evolutionary Computation, Vol. 1141 of LNCS, Springer Verlag: Berlin, Germany, 1996, pp. 32–41, ISBN 3-540-61723-XGoogle Scholar
  126. 126.
    H. Iba, “Evolutionary learning of communicating agents,” Inform. Sci., vol. 108, 1998Google Scholar
  127. 127.
    Iba, H. 1999

    “Evolving multiple agents by genetic programming,”

    Spector, L.Langdon, W.O’Reilly, U.-M.Angeline, P. eds. Advances in Genetic Programming 3The MIT PressCambridge, MA447466
    Google Scholar
  128. 128.
    I. Imam, (ed.), Intelligent Adaptive Agents. Papers from the 1996 AAAI Workshop. Technical Report WS-96-04, AAAI Press, 1996Google Scholar
  129. 129.
    A. Ito, “How do selfish agents learn to cooperate?,” in Artificial Life V: Proceedings of the Fifth International Workshop on the Synthesis and Simulation of Living Systems, MIT Press, pp. 185–192, 1997Google Scholar
  130. 130.
    T. Jansen and R. P. Wiegand, “Exploring the explorative advantage of the cooperative coevolutionary (1+1) EA,” in E. Cantu-Paz et al., (ed.), Prooceedings of the Genetic and Evolutionary Computation Conference (GECCO), Springer-Verlag, 2003Google Scholar
  131. 131.
    Jennings, N., Sycara, K., Wooldridge, M. 1998“A roadmap of agents research and development”Auton Agents Multi-Agent Syst.1738CrossRefGoogle Scholar
  132. 132.
    Jennings, N., Varga, L., Aarnts, R., Fuchs, J., Skarek, P. 1993“Transforming standalone expert systems into a community of cooperating agents”Int. J. Eng. Appl. Artif. Intell.6317331CrossRefGoogle Scholar
  133. 133.
    Jim, K.-C., Giles, C.L. 2000“Talking helps: Evolving communicating agents for the predator-prey pursuit problem”Artif. Life6237254CrossRefPubMedGoogle Scholar
  134. 134.
    H. Juille and J. Pollack, “Coevolving the “ideal” trainer: Application to the discovery of cellular automata rules”, in Proceedings of the Third Annual Genetic Programming Conference (GP-98), 1998Google Scholar
  135. 135.
    Kaelbling, L., Littman, M., Moore, A. 1996“Reinforcement learning: A survey”J. Artif. Intell. Res.4237285Google Scholar
  136. 136.
    S. Kapetanakis and D. Kudenko, Improving on the reinforcement learning of coordination in cooperative multi-agent systems, in Proceedings of the Second Symposium on Adaptive Agents and Multi-agent Systems (AISB02), 2002Google Scholar
  137. 137.
    S. Kapetanakis and D. Kudenko, “Reinforcement learning of coordination in cooperative multi-agent systems”, in Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI02), 2002Google Scholar
  138. 138.
    G. Kendall and G. Whitwell, “An evolutionary approach for the tuning of a chess evaluation function using population dynamics,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC-2001), IEEE Press, pp. 995–1002, 27–30, 2001Google Scholar
  139. 139.
    G. Kendall and M. Willdig, “An investigation of an adaptive poker player”, in Proceedings of the 14th Australian Joint Conference on Artificial Intelligence (AI’01), 2001Google Scholar
  140. 140.
    H. Kitano, M. Asada, Y. Kuniŷoshi, I. Noda, and E. Osawa, “RoboCup: The robot world cup initiative,” in W. L. Johnson and B. Hayes-Roth, (eds.), Proceedings of the First International Conference on Autonomous Agents (Agents’97), ACM Press: New York, 5-8, ISBN 0-89791-877-0, pp.’340-347, 1997Google Scholar
  141. 141.
    J. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, 1992Google Scholar
  142. 142.
    M. Lauer and M. Riedmiller, “An algorithm for distributed reinforcement learning in cooperative multi-agent systems,” in Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 535–542, 2000Google Scholar
  143. 143.
    L. R. Leerink, S. R. Schultz, and M. A. Jabri, “A reinforcement learning exploration strategy based on ant foraging mechanisms,” in Proceedings of the Sixth Australian Conference on Neural Networks, Sydney, Australia, 1995Google Scholar
  144. 144.
    Lesser, V. 1999“Cooperative multiagent systems : A personal view of the state of the art”IEEE Trans. Knowl. Data Eng.11133142CrossRefGoogle Scholar
  145. 145.
    V. Lesser, D. Corkill, and E. Durfee, An update on the distributed vehicle monitoring testbed, Technical Report UM-CS-1987-111, University of Massachessets Amherst, 1987Google Scholar
  146. 146.
    M. I. Lichbach, The Cooperator’s Dilemma, University of Michigan Press, 1996. ISBN 0472105728Google Scholar
  147. 147.
    M. Littman, “Markov games as a framework for multi-agent reinforcement learning”, in Proceedings of the 11th International Conference on Machine Learning (ML-94), Morgan Kaufmann: New Brunswick, NJ, pp. 157–163, 1994Google Scholar
  148. 148.
    M. Littman, “Friend-or-foe Q-learning in general-sum games,” in Proceedings of the Eighteenth International Conference on Machine Learning, Morgan Kaufmann, pp. 322–328, 2001Google Scholar
  149. 149.
    A. Lubberts and R. Miikkulainen, “Co-evolving a go-playing neural network,” in Coevolution: Turning Adaptive Algorithms upon Themselves, (Birds-on-a-Feather Workshop, Genetic and Evolutionary Computation Conference), 2001Google Scholar
  150. 150.
    Luck, M. d’Inverno, M. Fisher, M. FoMAS’97 Contributors1998“Foundations of multi-agent systems: Techniques, tools and theory”Knowl. Eng. Rev.13297302CrossRefGoogle Scholar
  151. 151.
    S. Luke, “Genetic programming produced competitive soccer softbot teams for RoboCup97,” in J. R. Koza et al, (ed.), Genetic Programming 1998: Proceedings of the Third Annual Conference, Morgan Kaufmann, pp. 214–222, 1998Google Scholar
  152. 152.
    S. Luke, C. Hohn, J. Farris, G. Jackson, and J. Hendler, “Co-evolving soccer softbot team coordination with genetic programming,” in Proceedings of the First International Workshop on RoboCup, at’the International Joint Conference on Artificial Intelligence, Nagoya, Japan, 1997Google Scholar
  153. 153.
    S. Luke and L. Spector, “Evolving teamwork and coordination with genetic programming,” in J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, (eds.), Genetic Programming 1996: Proceedings of the First Annual Conference, MIT Press: Stanford University, CA, USA, pp. 150–156, 28-31 1996Google Scholar
  154. 154.
    S. Luke, K. Sullivan, G. C. Balan, and L. Panait, Tunably decentralized algorithms for cooperative target observation, Technical Report GMU-CS-TR-2004-1, Department of Computer Science, George Mason University, 2004Google Scholar
  155. 155.
    S. Luke and R. P. Wiegand, “Guaranteeing coevolutionary objective measures”, in Poli et al. [201], pp. 237–251Google Scholar
  156. 156.
    S. Mahadevan and J. Connell, “Automatic programming of behavior-based robots using reinforcement learning,” in National Conference on Artificial Intelligence, pp. 768–773, 1991Google Scholar
  157. 157.
    Makar, R., Mahadevan, S., Ghavamzadeh, M. 2001

    “Hierarchical multi-agent reinforcement learning”

    Müller, J.P.Andre, E.Sen, S.Frasson, C. eds. Proceedings of the Fifth International Conference on Autonomous AgentsACM PressMontreal, Canada246253
    Google Scholar
  158. 158.
    Mataric, M. 1994Interaction and Intelligent BehaviorMassachusetts Institute of TechnologyCambridge, MAPhD thesis, Also Technical Report AITR-1495Google Scholar
  159. 159.
    M. Mataric, “Learning to behave socially,” in Third International Conference on Simulation of Adaptive Behavior, 1994Google Scholar
  160. 160.
    M. Mataric, “Reward functions for accelerated learning,” in International Conference on Machine Learning, pp. 181–189, 1994Google Scholar
  161. 161.
    M. Mataric, “Reinforcement learning in the multi-robot domain,” Auton. Robots, vol. 4, no. 1, pp.’73–83, 1997Google Scholar
  162. 162.
    M. Mataric, “Using communication to reduce locality in distributed multi-agent learning,” Joint Special Issue on Learn Auton. Robots, Machine Learn, vol. 31, nos. 1-3, pp. 141–167, and Auton. Robots, vol. 5, nos. 3-4, pp. 335–354, Jul/Aug 1998Google Scholar
  163. 163.
    M. Mataric, M. Nilsson, and K. Simsarian, “Cooperative multi-robot box-pushing,” in Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems, pp. 556–561, 1995Google Scholar
  164. 164.
    Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs (3rd edn.), Springer-Verlag: Berlin, 1996Google Scholar
  165. 165.
    T. Miconi, “A collective genetic algorithm”, in E. Cantu-Paz et al., (ed.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 876–883, 2001Google Scholar
  166. 166.
    T. Miconi, “When evolving populations is better than coevolving individuals: The blind mice problem,” in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003Google Scholar
  167. 167.
    M. Mitchell, J. Crutchfield, and R. Das, “Evolving cellular automata with genetic algorithms: A’review of recent work,” in Proceedings of the First International Conference on Evolutionary Computation and its Applications (EvCA’96), 1996Google Scholar
  168. 168.
    N. Monekosso, P. Remagnino, and A. Szarowicz, “An improved Q-learning algorithm using synthetic pheromones,” in E. N. B. Dunin-Keplicz, (ed.), From Theory to Practice in Multi-Agent Systems, Second International Workshop of Central and Eastern Europe on Multi-Agent Systems, CEEMAS 2001 Cracow, Poland, September 26–29, 2001. Revised Papers, Lecture Notes in Artificial Intelligence LNAI-2296, Springer-Verlag, 2002Google Scholar
  169. 169.
    N. D. Monekosso and P. Remagnino, “Phe-Q: A pheromone based Q-learning,” in Australian Joint Conference on Artificial Intelligence, pp. 345–355, 2001Google Scholar
  170. 170.
    N. D. Monekosso and P. Remagnino, “An analysis of the pheromone Q-learning algorithm,” in’Proceedings of the VIII Iberoamerican Conference on Artificial Intelligence IBERAMIA-02, pp. 224–232, 2002Google Scholar
  171. 171.
    N. D. Monekosso, P. Remagnino, and A. Szarowicz, “An improved Q-learning algorithm using synthetic pheromones,” in Proceedings of the Second Workshop of Central and Eastern Europe on Multi-Agent Systems CEEMAS-01, pp. 197–206, 2001Google Scholar
  172. 172.
    J. Moody, Y. Liu, M. Saffell, and K. Youn, “Stochastic direct reinforcement: Application to simple games with recurrence,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004Google Scholar
  173. 173.
    R. Mukherjee and S. Sen, “Towards a pareto-optimal solution in general-sum games,” in Agents-2001 Workshop on Learning Agents, 2001Google Scholar
  174. 174.
    U. Mukhopadjyay, L. Stephens, and M. Huhns, “An intelligent system for document retrieval in distributed office environment,” J. Am. Soc. Inform Sci., vol. 37, 1986Google Scholar
  175. 175.
    Muller, J., Pischel, M. 1994“An architecture for dynamically interacting agents”J. Intell. Coop. Inform. Syst.32545CrossRefGoogle Scholar
  176. 176.
    M. Mundhe and S. Sen, “Evaluating concurrent reinforcement learners,” in Proceedings of the International Conference on Multiagent System, 2000Google Scholar
  177. 177.
    M. Mundhe and S. Sen, “Evolving agent societies that avoid social dilemmas,” in D. Whitley, D.’Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, and H.-G. Beyer, (eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), Morgan Kaufmann: Las Vegas, Nevada, USA, 10-12 2000, pp. 809–816, ISBN 1-55860-708-0Google Scholar
  178. 178.
    Y. Nagayuki, S. Ishii, and K. Doya, “Multi-agent reinforcement learning: An approach based on the other agent’s internal model,” in Proceedings of the International Conference on Multi-Agent Systems (ICMAS-00), 2000Google Scholar
  179. 179.
    M. V. Nagendra-Prasad, Learning Situation-Specific Control in Multi-Agent Systems, PhD thesis, University of Massachusetts Amherst, 1997Google Scholar
  180. 180.
    R. Nair, D. Pynadath, M. Yokoo, M. Tambe, and S. Marsella, “Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings,” in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003Google Scholar
  181. 181.
    Nowak, M., Sigmund, K. 1998“Evolution of indirect reciprocity by image scoring/the dynamics of indirect reciprocity”Nature393573577CrossRefPubMedGoogle Scholar
  182. 182.
    A. Nowe, K. Verbeeck, and T. Lenaerts, Learning agents in a homo egualis society, Technical report, Computational Modeling Lab - VUB, March 2001Google Scholar
  183. 183.
    L. Nunes and E. Oliveira, “Learning from multiple sources,” in AAMAS-2004- Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  184. 184.
    T. Ohko, K. Hiraki, and Y. Arzai, “Addressee learning and message interception for communication load reduction in multiple robots environments,” in G. Weiß, (ed.), Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agent Environments, Lecture Notes in Artificial Intelligence 1221, Springer-Verlag, 1997Google Scholar
  185. 185.
    E. Ostergaard, G. Sukhatme, and M. Mataric, “Emergent bucket brigading - a simple mechanism for improving performance in multi-robot constrainedspace foraging tasks,” in Proceedings of the Fifth International Conference on Autonomous Agents, 2001Google Scholar
  186. 186.
    L. Pagie and M. Mitchell, “A comparison of evolutionary and coevolutionary search,” in R. K. Belew and H. Juillè, (eds.), Coevolution: Turning Adaptive Algorithms upon Themselves, San Francisco, California, USA, pp. 20–25, 7 2001Google Scholar
  187. 187.
    L. Panait and S. Luke, “Ant foraging revisited,” in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), 2004Google Scholar
  188. 188.
    L. Panait and S. Luke, “Learning ant foraging behaviors,” in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), 2004Google Scholar
  189. 189.
    L. Panait and S. Luke, “A pheromone-based utility model for collaborative foraging,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  190. 190.
    L. Panait, R. P. Wiegand, and S. Luke, “A sensitivity analysis of a cooperative coevolutionary algorithm biased for optimization,” in Genetic and Evolutionary Computation Conference - GECCO-2004, Springer, 2004Google Scholar
  191. 191.
    L. Panait, R. P. Wiegand, and S. Luke, “A visual demonstration of convergence properties of cooperative coevolution,” in Parallel Problem Solving from Nature - PPSN-2004, Springer, 2004Google Scholar
  192. 192.
    L. A. Panait, R. P. Wiegand, and S. Luke, “Improving coevolutionary search for optimal multiagent behaviors”, in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003Google Scholar
  193. 193.
    Papadimitriou, C., Tsitsiklis, J. 1987“Complexity of markov decision processes”Math. Operat. Res.12441450Google Scholar
  194. 194.
    L. Parker, “Current state of the art in distributed autonomous mobile robotics,” in L. Parker, G.’Bekey, and J. Barhen, (eds.), Distributed Autonomous Robotic Systems 4, Springer-Verlag, pp. 3–12, 2000Google Scholar
  195. 195.
    L. Parker, “Multi-robot learning in a cooperative observation task,” in Proceedings of Fifth International Symposium on Distributed Autonomous Robotic Systems (DARS 2000), 2000Google Scholar
  196. 196.
    L. Parker, “Distributed algorithms for multi-robot observation of multiple moving targets,” Auton Robots, vol. 12 no. 3, 2002Google Scholar
  197. 197.
    L. Parker, C. Touzet, and F. Fernandez, “Techniques for learning in multi-robot teams,” in T. Balch and L. Parker, (eds.), Robot Teams: From Diversity to Polymorphism, AK Peters, 2001Google Scholar
  198. 198.
    M. Peceny, G. Weiß, and W. Brauer, Verteiltes maschinelles lernen in fertigungsumgebungen, Technical Report FKI-218-96, Institut fur Informatik, Technische Universitat Munchen, 1996Google Scholar
  199. 199.
    M. Peeters, K. Verbeeck, and A. Nowe, “Multi-agent learning in conflicting multi-level games with incomplete information,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004Google Scholar
  200. 200.
    L. Peshkin, K.-E. Kim, N. Meuleau, and L. Kaelbling, “Learning to cooperate via policy search,” in Sixteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, 2000, pp. 307–314Google Scholar
  201. 201.
    R. Poli, J. Rowe, and K. D. Jong, (eds.), Foundations of Genetic Algorithms (FOGA) VII, 2002, Morgan KaufmannGoogle Scholar
  202. 202.
    Pollack, J., Blair, A. 1998“Coevolution in the successful learning of backgammon strategy”Machine Learn.32225240CrossRefGoogle Scholar
  203. 203.
    Pollack, J., Blair, A., Land, M. 1997

    “Coevolution of a backgammon player”

    Langton, C.G.Shimohara, K. eds. Artificial Life V: Proc. of the Fifth Int. Workshop on the Synthesis and Simulation of Living SystemsThe MIT PressCambridge, MA9298
    Google Scholar
  204. 204.
    E. Popovici and K. DeJong, “Understanding competitive co-evolutionary dynamics via fitness landscapes,” in Artificial Multiagent Symposium, Part of the 2004 AAAI Fall Symposium on Artificial Intelligence, 2004Google Scholar
  205. 205.
    Potter, M. 1997The Design and Analysis of a Computational Model of Cooperative Coevolution, PhD thesisGeorge Mason UniversityFairfax, VirginiaGoogle Scholar
  206. 206.
    M. Potter and K. De Jong, “A cooperative coevolutionary approach to function optimization,” in Y. Davidor and H.-P. Schwefel, (eds.), Proceedings of the Third International Conference on Parallel Problem Solving from Nature (PPSN III), Springer-Verlag, pp. 249–257, 1994Google Scholar
  207. 207.
    Potter, M., Jong, K. 2000“Cooperative coevolution: An architecture for evolving coadapted subcomponents”Evol. Comput.8129CrossRefPubMedGoogle Scholar
  208. 208.
    M. Potter, K. De Jong, and J. J. Grefenstette, “A coevolutionary approach to learning sequential decision rules,” in Proceedings from the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp. 366–372, 1995Google Scholar
  209. 209.
    M. Potter, L. Meeden, and A. Schultz, “Heterogeneity in the coevolved behaviors of mobile robots: The emergence of specialists,” in Proceedings of The Seventeenth International Conference on Artificial Intelligence (IJCAI-2001), 2001Google Scholar
  210. 210.
    N. Puppala, S. Sen, and M. Gordin, “Shared memory based cooperative coevolution,” in Proceedings of the 1998 IEEE World Congress on Computational Intelligence, IEEE Press: Anchorage, Alaska, USA, pp. 570–574, 1998Google Scholar
  211. 211.
    M. Quinn, “A comparison of approaches to the evolution of homogeneous multi-robot teams,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC2001), IEEE Press: COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27–30 2001, pp. 128–135. ISBN 0-7803-6658-1Google Scholar
  212. 212.
    M. Quinn, “Evolving communication without dedicated communication channels,” in Advances in Artificial Life: Sixth European Conference on Artificial Life (ECAL01), 2001Google Scholar
  213. 213.
    M. Quinn, L. Smith, G. Mayley, and P. Husbands, Evolving formation movement for a homogeneous multi-robot system: Teamwork and role-allocation with real robots, Cognitive Science Research Paper 515. School of Cognitive and Computing Sciences, University of Sussex, Brighton, BN1 9QG. ISSN 1350–3162, 2002Google Scholar
  214. 214.
    C. Reynolds, “An evolved, vision-based behavioral model of coordinated group motion,” in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior (SAB92), pp. 384–392, 1993Google Scholar
  215. 215.
    C. Reynolds, “Competition, coevolution and the game of tag,” in R. A. Brooks and P. Maes, (eds.), Artificial Life IV, Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems., MIT Press, pp. 59–69, 1994Google Scholar
  216. 216.
    Reynolds, C.W. 1987“Flocks, herds, and schools: a distributed behavioral model”Comput. Graph.212534Google Scholar
  217. 217.
    P. Riley and M. Veloso, “On behavior classification in adversarial environments,” in L. Parker, G.’Bekey, and J. Barhen (eds.), Distributed Autonomous Robotic Systems 4, Springer-Verlag, pp. 371–380, 2000Google Scholar
  218. 218.
    A. Robinson and L. Spector, “Using genetic programming with multiple data types and automatic modularization to evolve decentralized and coordinated navigation in multi-agent systems,” in In’Late-Breaking Papers of the Genetic and Evolutionary Computation Conference (GECCO-2002), The International Society for Genetic and Evolutionary Computation, 2002Google Scholar
  219. 219.
    Rosin, C., Belew, R. 1997“New methods for competitive coevolution”Evol. Comput.5129PubMedGoogle Scholar
  220. 220.
    R. Salustowicz, M. Wiering, and J. Schmidhuber, Learning team strategies with multiple policy-sharing agents: A soccer case study, Technical report, ISDIA, Corso Elvezia 36, 6900 Lugano, Switzerland, 1997Google Scholar
  221. 221.
    Salustowicz, R., Wiering, M., Schmidhuber, J. 1998“Learning team strategies: Soccer case studies”Machine Learn.33263282CrossRefGoogle Scholar
  222. 222.
    Samuel, A. 1994“Some studies in machine learning using the game of checkers”IBM J. Res. Develop.3210229Google Scholar
  223. 223.
    T. Sandholm and R. H. Crites, “On multiagent Q-learning in a semi-competitive domain,” in’Adaption and Learning in Multi-Agent Systems, pp. 191–205, 1995Google Scholar
  224. 224.
    H. Santana, G. Ramalho, V. Corruble, and B. Ratitch, “Multi-agent patrolling with reinforcement learning,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  225. 225.
    G. Saunders and J. Pollack, “The evolution of communication schemes over continuous channels,” in’From Animals to Animats 4 - Proceedings of the Fourth International Conference on Adaptive Behaviour, 1996Google Scholar
  226. 226.
    J. Sauter, R. S. Matthews, H. Van Dyke Parunak, and S. Brueckner, “Evolving adaptive pheromone path planning mechanisms,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 434–440, 2002Google Scholar
  227. 227.
    J. Sauter, H. Van Dyke Parunak, S. Brueckner, and R. Matthews, “Tuning synthetic pheromones with evolutionary computing,” in R. E. Smith, C. Bonacina, C. Hoile, and P. Marrow, (eds.), Evolutionary Computation and Multi-Agent Systems (ECOMAS), San Francisco, California, USA, 7 pp. 321–324, 2001Google Scholar
  228. 228.
    J. Schmidhuber, “Realistic multi-agent reinforcement learning,” in Learning in Distributed Artificial Intelligence Systems, Working Notes of the 1996 ECAI Workshop, 1996Google Scholar
  229. 229.
    J. Schmidhuber and J. Zhao, “Multi-agent learning with the success-story algorithm,” in ECAI Workshop LDAIS/ICMAS Workshop LIOME, pp. 82–93, 1996Google Scholar
  230. 230.
    J. Schneider, W.-K. Wong, A. Moore, and M. Riedmiller, “Distributed value functions,” in Proceedings of the Sixteenth International Conference on Machine Learning, pp. 371–378, 1999Google Scholar
  231. 231.
    A. Schultz, J. Grefenstette, and W. Adams, “Robo-shepherd: Learning complex robotic behaviors,” in Robotics and Manufacturing: Recent Trends in Research and Applications. Vol. 6, ASME Press, pp. 763–768, 1996Google Scholar
  232. 232.
    U. M. Schwuttke and A. G. Quan, “Enhancing performance of cooperating agents in realtime diagnostic systems”, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (IJCAI-93), 1993Google Scholar
  233. 233.
    M. Sekaran and S. Sen, “To help or not to help”, in Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society, Pittsburgh, PA, pp. 736–741, 1995Google Scholar
  234. 234.
    S. Sen, “Multiagent systems: Milestones and new horizons”, Trends Cognitive Sci., vol. 1, no. 9, pp.’334–339, 1997Google Scholar
  235. 235.
    S. Sen, “Special issue on evolution and learning in multiagent systems,” Int. J. Human-Comput. Stud., vol. 48, no. 1, 1998Google Scholar
  236. 236.
    S. Sen and M. Sekaran, “Using reciprocity to adapt to others”, in G. Weiß and S. Sen (eds.), International Joint Conference on Artificial Intelligence Workshop on Adaptation and Learning in Multiagent Sytems, Lecture Notes in Artificial Intelligence, Springer-Verlag, pp. 206–217, 1995Google Scholar
  237. 237.
    S. Sen and M. Sekaran, “Multiagent coordination with learning classifier systems”, in G. Weiß and S.’Sen, (eds.), Proceedings of the IJCAI Workshop on Adaption and Learning in Multi-Agent Systems, Volume 1042, Springer Verlag, pp. 218–233, 1996. ISBN 3-540-60923-7Google Scholar
  238. 238.
    Sen, S., Sekaran, M. 1998“Individual learning of coordination knowledge”J. Exp. Theo. Artif. Intel.10333356CrossRefGoogle Scholar
  239. 239.
    S. Sen, M. Sekaran, and J. Hale, “Learning to coordinate without sharing information”, in Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 426–431, 1994Google Scholar
  240. 240.
    Y. Shoham, R. Powers, and T. Grenager, “On the agenda(s) of research on multi-agent learning,” in’Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04–02, 2004Google Scholar
  241. 241.
    R. Smith and B. Gray, Co-adaptive genetic algorithms: An example in othello strategy, Technical Report TCGA 94002, University of Alabama, Department of Engineering Science and Mechanics, 1993Google Scholar
  242. 242.
    L. Spector and J. Klein, “Evolutionary dynamics discovered via visualization in the breve simulation environment,” in Workshop Proceedings of the 8th International Conference on the Simulation and Synthesis of Living Systems, pp. 163–170, 2002Google Scholar
  243. 243.
    L. Spector, J. Klein, C. Perry, and M. Feinstein, “Emergence of collective behavior in evolving populations of flying agents,” in E. Cantu-Paz et al., (ed.), Prooceedings of the Genetic and Evolutionary Computation Conference (GECCO). Springer-Verlag, 2003Google Scholar
  244. 244.
    R. Steeb, S. Cammarata, F. Hayes-Roth, P. Thorndyke, and R. Wesson, “Distributed intelligence for air fleet control,” in A. Bond and L. Gasser (eds.), Readings in Distributed Artificial Intelligence, Morgan Kaufmann Publishers, pp. 90–101, 1988Google Scholar
  245. 245.
    Steels, L. 1995“A self-organizing spatial vocabulary”Artif. Life2319332PubMedGoogle Scholar
  246. 246.
    L. Steels, “Emergent adaptive lexicons,” in P. Maes, (ed.), Proceedings of the Simulation of Adaptive Behavior Conference. MIT Press, 1996Google Scholar
  247. 247.
    L. Steels, “Self-organising vocabularies,” in Proceedings of Artificial Life V, 1996Google Scholar
  248. 248.
    Steels, L. 1996

    “The spontaneous self-organization of an adaptive language”

    Muggleton, S. eds. Machine Intelligence 15Oxford University PressOxford, UK
    Google Scholar
  249. 249.
    L. Steels, “Synthesising the origins of language and meaning using co-evolution, self-organisation and level formation,” in J. Hurford, C. Knight, and M. Studdert-Kennedy (eds.), Approaches to the Evolution of Language: Social and Cognitive Bases, Edinburgh University Press, 1997Google Scholar
  250. 250.
    Steels, L. 2000“The puzzle of language evolution”Kognitionswissenschaft8143150CrossRefGoogle Scholar
  251. 251.
    L. Steels and F. Kaplan, “Collective learning and semiotic dynamics,” in Proceedings of the European Conference on Artificial Life, pp. 679–688, 1999Google Scholar
  252. 252.
    P. Stone, “Layered learning in multiagent systems,” in Proceedings of National Conference on Artificial Intelligence AAAI/IAAI, 1997Google Scholar
  253. 253.
    P. Stone, “Layered Learning in Multi-Agent Systems,” PhD thesis, Carnegie Mellon University, 1998Google Scholar
  254. 254.
    P. Stone and R. Sutton, “Keepaway soccer: A machine learning testbed,” in A. Birk, S. Coradeschi, and S. Tadokoro, (eds.), RoboCup 2001: Robot Soccer World Cup V, volume 2377 of Lecture Notes in Computer Science, Springer, pp. 214–223, 2002. ISBN 3-540-43912-9Google Scholar
  255. 255.
    Stone, P., Veloso, M.M. 2000“Multiagent systems: A survey from a machine learning perspective”Auton. Robots8345383CrossRefGoogle Scholar
  256. 256.
    N. Sturtevant and R. Korf, “On pruning techniques for multi-player games,” in Proceedings of National Conference on Artificial Intelligence (AAAI), pp. 201–207, 2000Google Scholar
  257. 257.
    D. Subramanian, P. Druschel, and J. Chen, “Ants and reinforcement learning: A case study in routing in dynamic networks,” in Proceedings of Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97), pp. 832–839, 1997Google Scholar
  258. 258.
    N. Suematsu and A. Hayashi, “A multiagent reinforcement learning algorithm using extended optimal response,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 370–377, 2002Google Scholar
  259. 259.
    D. Suryadi and P. J. Gmytrasiewicz, “Learning models of other agents using influence diagrams,” in’Preceedings of the 1999 International Conference on User Modeling, pp. 223–232, 1999Google Scholar
  260. 260.
    Sutton, R. 1998“Learning to predict by the methods of temporal differences”Machine Learn.3944Google Scholar
  261. 261.
    R. Sutton and A. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998Google Scholar
  262. 262.
    J. Svennebring and S. Koenig, “Trail-laying robots for robust terrain coverage,” in Proceedings of the International Conference on Robotics and Automation (ICRA-03), 2003Google Scholar
  263. 263.
    P. ’t Hoen and K. Tuyls, “Analyzing multi-agent reinforcement learning using evolutionary dynamics,” in Proceedings of the 15th European Conference on Machine Learning (ECML), 2004Google Scholar
  264. 264.
    M. Tambe, “Recursive agent and agent-group tracking in a real-time dynamic environment,” in V.’Lesser and L. Gasser (eds.), Proceedings of the First International Conference on Multiagent Systems (ICMAS-95). AAAI Press, 1995Google Scholar
  265. 265.
    Tan, M. 1993

    “Multi-agent reinforcement learning: Independent vs. cooperative learning”

    Huhns, M.N.Singh, M.P. eds. Readings in AgentsMorgan KaufmannSan Francisco, CA, USA487494
    Google Scholar
  266. 266.
    P. Tangamchit, J. Dolan, and P. Khosla, “The necessity of average rewards in cooperative multirobot learning,” in Proceedings of IEEE Conference on Robotics and Automation, 2002Google Scholar
  267. 267.
    Tesauro, G. 1995“Temporal difference learning and TD-gammon”Commun. ACM385868CrossRefGoogle Scholar
  268. 268.
    Tesauro, G., Kephart, J.O. 2002“Pricing in agent economies using multi-agent Q-learning”Auton. Agents Multi-Agent Syst.8289304CrossRefGoogle Scholar
  269. 269.
    S. Thrun, “Learning to play the game of chess,” in G. Tesauro, D. Touretzky, and T. Leen, (eds.), Advances in Neural Information Processing Systems 7, The MIT Press: Cambridge, MA, pp. 1069–1076, 1995Google Scholar
  270. 270.
    K. Tumer, A. K. Agogino, and D. H. Wolpert, “Learning sequences of actions in collectives of autonomous agents,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 378–385, 2002Google Scholar
  271. 271.
    K. Tuyls, K. Verbeeck, and T. Lenaerts, “A selection-mutation model for Q-learning in multiagent systems,” in AAMAS-2003 — Proceedings of the Second International Joint Conference on Autonomous Agents and Multi Agent Systems, 2003Google Scholar
  272. 272.
    W. Uther and M. Veloso, “Adversarial reinforcement learning. Technical Report CMU-CS-03-107, School of Computer Science, Carnegie Mellon University, 2003Google Scholar
  273. 273.
    H. Van Dyke Parunak, “Applications of distributed artificial intelligence in industry,” in G. M. P. O’Hare and N. R. Jennings, (eds.), Foundations of Distributed AI. John Wiley & Sons, 1996Google Scholar
  274. 274.
    Varga, L.Z., Jennings, N.R., Cockburn, D. 1994“Integrating intelligent systems into a cooperating community for electricity distribution management”Int. J. Expert Syst. Appl.7563579CrossRefGoogle Scholar
  275. 275.
    J. Vidal and E. Durfee, “Agents learning about agents: A framework and analysis,” in Working Notes of AAAI-97 Workshop on Multiagent Learning, 1997Google Scholar
  276. 276.
    J. Vidal and E. Durfee, “The moving target function problem in multiagent learning,” in Proceedings of the Third Annual Conference on Multi-Agent Systems, 1998Google Scholar
  277. 277.
    J. Vidal and E. Durfee, “Predicting the expected behavior of agents that learn about agents: The CLRI framework,” Autonomous Agents and Multi-Agent Systems, January 2003Google Scholar
  278. 278.
    K. Wagner, “Cooperative strategies and the evolution of communication,” Artif. Life, vol. 6, no. 2, pp. 149–179, Spring 2000Google Scholar
  279. 279.
    X. Wang and T. Sandholm, “Reinforcement learning to play an optimal Nash equilibrium in team Markov games,” in Advances in Neural Information Processing Systems (NIPS-2002), 2002Google Scholar
  280. 280.
    R. Watson and J. Pollack, “Coevolutionary dynamics in a minimal substrate,” in E. Cantu-Paz et al, (eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), 2001Google Scholar
  281. 281.
    R. Weihmayer and H. Velthuijsen, “Application of distributed AI and cooperative problem solving to telecommunications,” in J. Liebowitz and D. Prereau, (eds.), AI Approaches to Telecommunications and Network Management, IOS Press, 1994Google Scholar
  282. 282.
    M. Weinberg and J. Rosenschein, “Best-response multiagent learning in non-stationary environments,” in AAMAS-2004 — Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004Google Scholar
  283. 283.
    G. Weiß, Some studies in distributed machine learning and organizational design. Technical Report FKI-189-94, Institut f´′ur Informatik, TU München, 1994Google Scholar
  284. 284.
    Weiß, G. 1995Distributed Machine LearningInfix VerlagSankt AugustinGoogle Scholar
  285. 285.
    G. Weiß, ed., Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agent Environments, Number 1221 in Lecture Notes in Artificial Intelligence, Springer-Verlag, 1997Google Scholar
  286. 286.
    G. Weiß (1998). “Special issue on learning in distributed artificial intelligence systems”. J. Exp. Theo. Artif. Intell. 10(3).Google Scholar
  287. 287.
    G. Weiß, ed., Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, 1999Google Scholar
  288. 288.
    G. Weiß and P. Dillenbourg, “What is ‘multi’ in multi-agent learning?” in P. Dillenbourg, (ed.), Collaborative Learning, Cognitive and Computational Approaches, Pergamon Press, pp. 64–80, 1999Google Scholar
  289. 289.
    G. Weiß and S. Sen (eds.), Adaptation and Learning in Multiagent Systems,” Lecture Notes in Artificial Intelligence Vol. 1042, Springer-Verlag, 1996Google Scholar
  290. 290.
    Wellman, M., Hu, J. 1998“Conjectural equilibrium in multiagent learning”Machine Learn.33179200CrossRefGoogle Scholar
  291. 291.
    Werfel, J., Mitchell, M., Crutchfield, J.P. November 2000“Resource sharing and coevolution in evolving cellular automata”IEEE Trans. Evol. Comput.4388Google Scholar
  292. 292.
    B. B. Werger and M. Mataric, “Exploiting embodiment in multi-robot teams, Technical Report IRIS-99-378, University of Southern California, Institute for Robotics and Intelligent Systems, 1999Google Scholar
  293. 293.
    G. M. Werner and M. G. Dyer, “Evolution of herding behavior in artificial animals,” in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior (SAB92), 1993Google Scholar
  294. 294.
    White, T., Pagurek, B., Oppacher, F. 1998

    “ASGA: Improving the ant system by integration with genetic algorithms”

    Koza, J.R.Banzhaf, W.Chellapilla, K.Deb, K.Dorigo, M.Fogel, D.B.Garzon, M.H.Goldberg, D.E.Iba, H.Riolo, R. eds. Genetic Programming 1998: Proceedings of the Third Annual ConferenceMorgan Kaufmann: University of WisconsinMadison, Wisconsin, USA2225610–617
    Google Scholar
  295. 295.
    S. Whiteson and P. Stone, “Concurrent layered learning,” in AAMAS-2003 - Proceedings of the Second International Joint Conference on Autonomous Agents and Multi Agent Systems, 2003Google Scholar
  296. 296.
    R. P. Wiegand, Analysis of Cooperative Coevolutionary Algorithms, PhD thesis, Department of Computer Science, George Mason University, 2003Google Scholar
  297. 297.
    R. P. Wiegand, W. Liles, and K. De Jong, “An empirical analysis of collaboration methods in cooperative coevolutionary algorithms,” in E. Cantu-Paz et al., (ed.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 1235–1242, 2001Google Scholar
  298. 298.
    R. P. Wiegand, W. Liles, and K. De Jong, “Analyzing cooperative coevolution with evolutionary game theory,” in D. Fogel, (ed.), Proceedings of Congress on Evolutionary Computation (CEC-02), IEEE Press, pp. 1600–1605, 2002Google Scholar
  299. 299.
    R. P. Wiegand, W. Liles, and K. De Jong, “Modeling variation in cooperative coevolution using evolutionary game theory,” in Poli et al. [201], pp. 231–248Google Scholar
  300. 300.
    R. P. Wiegand and J. Sarma, “Spatial embedding and loss of gradient in cooperative coevolutionary algorithms,” in Parallel Problem Solving from Nature - PPSN-2004, Springer, 2004Google Scholar
  301. 301.
    Wiering, M., Salustowicz, R., Schmidhuber, J. 1999“Reinforcement learning soccer teams with incomplete world models”J. Auton. Robots77788CrossRefGoogle Scholar
  302. 302.
    Williams, A. 2004“Learning to share meaning in a multi-agent system”Auton. Agents Multi-Agent Syst.8165193CrossRefGoogle Scholar
  303. 303.
    E. Wilson, Sociobiology: The New Synthesis, Belknap Press, 1975Google Scholar
  304. 304.
    Wolpert, D.H., Tumer, K. 2001“Optimal payoff functions for members of collectives”Adv. Complex Syst.4265279CrossRefGoogle Scholar
  305. 305.
    D. H. Wolpert, K. Tumer, and J. Frank, “Using collective intelligence to route internet traffic,” in’Advances in Neural Information Processing Systems-11, Denver, pp. 952–958, 1998Google Scholar
  306. 306.
    D. H. Wolpert, K. R. Wheller, and K. Tumer, “General principles of learning-based multi-agent systems,” in O. Etzioni, J. P. Müller, and J. M. Bradshaw, (eds.), Proceedings of the Third International Conference on Autonomous Agents (Agents’99), ACM Press: Seattle, WA, USA, pp. 77–83, 1999Google Scholar
  307. 307.
    M. Wooldridge, S. Bussmann, and M. Klosterberg, “Production sequencing as negotiation,” in’Proceedings of the First International Conference on the Practical Application of Intelligent Agents and Multi-Agent Technology (PAAM-96), 1996Google Scholar
  308. 308.
    A. Wu, A. Schultz, and A. Agah, “Evolving control for distributed micro air vehicles,” in IEEE Computational Intelligence in Robotics and Automation Engineers Conference, 1999Google Scholar
  309. 309.
    H. Yanco and L. Stein, “An adaptive communication protocol for cooperating mobile robots,” in’From Animals to Animats: International Conference on Simulation of Adaptive Behavior, pp. 478–485, 1993Google Scholar
  310. 310.
    N. Zaera, D. Cliff, and J. Bruten, (Not) Evolving collective behaviours in synthetic fish, Technical Report HPL-96-04, Hewlett-Packard Laboratories, 1996Google Scholar
  311. 311.
    B. Zhang and D. Cho, “Coevolutionary fitness switching: Learning complex collective behaviors using genetic programming,” in Advances in Genetic Programming III, MIT Press, 1998, pp. 425–445Google Scholar
  312. 312.
    Zhao, J., Schmidhuber, J. 1996

    “Incremental self-improvement for life-time multi-agent reinforcement learning”

    Maes, P.Mataric, M.Meyer, J.-A.Pollack, J.Wilson, S.W. eds. Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior: From Animals to Animats 4MIT PressCape Code, USA913516–525, ISBN 0-262-63178-4
    Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Department of Computer ScienceGeorge Mason UniversityFairfaxUSA

Personalised recommendations