Skip to main content
SpringerLink
Log in

A Survey of Collectives

  • Chapter
Collectives and the Design of Complex Systems

Summary

Due to the increasing sophistication and miniaturization of computational components, complex, distributed systems of interacting agents are becoming ubiquitous. Such systems, where each agent aims to optimize its own performance, but there is a well-defined set of system-level performance criteria, are called collectives. The fundamental problem in analyzing and designing such systems is in determining how the combined actions of a large number of agents lead to “coordinated” behavior on the global scale. Examples of artificial systems that exhibit such behavior include packet routing across a data network, control of an array of communication satellites, coordination of multiple rovers, and dynamic job scheduling across a distributed computer grid. Examples of natural systems include ecosystems, economies, and the organelles within a living cell.

No current scientific discipline provides a thorough understanding of the relation between the structure of collectives and how well they meet their overall performance criteria. Although still very young, research on collectives has resulted in successes in both understanding and designing such systems. It is expected that as it matures and draws on other disciplines related to collectives, this field will greatly expand the range of computationally addressable tasks. Moreover, in addition to drawing on them, such a fully developed field of collective intelligence may provide insight into already established scientific fields, such as mechanism design, economics, game theory, and population biology. This chapter provides a survey of the emerging science of collectives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. H. Abelson, D. Allen, D. Coore, C. Hanson, G. Homsy, T. F. Knight, Jr., R. Nagpal, E. Rauch, G. J. Sussman, and R. Weiss. Amorphous computing. Communications of the ACM, 43(5), May 2000.

    Google Scholar 

  2. H. Abelson and N. Forbes. Morphous-computing techniques may lead to intelligent materials. Computers in Physics, 12(6):520–2, 1998.

    Article  Google Scholar 

  3. M. R. Anderson and T. W. Sandholm. Leveled commitment contracts with myopic and strategic agents. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 36–45, 1998.

    Google Scholar 

  4. K. Arrow and G. Debreu. The existence of an equilibrium for a competitive equilibrium. Econometrica, 22:265–90, 1954.

    Article  MathSciNet  Google Scholar 

  5. W. B. Arthur. Complexity in economic theory: Inductive reasoning and bounded rationality. The American Economic Review, 84(2):406–11, May 1994.

    Google Scholar 

  6. W. Ashcroft and N. D. Mermin. Solid State Physics. W. B. Saunders, Philadelphia, 1976.

    Google Scholar 

  7. J.J. Astrom and B. Wittenmark. Adaptive Control. Addison-Wesley, 1994.

    Google Scholar 

  8. C. G. Atkeson. Nonparametric model-based reinforcement learning. In Advances in Neural Information Processing Systems—10, pages 1008–14. MIT Press, 1998.

    Google Scholar 

  9. C. G. Atkeson, S. A. Schaal, and A. W. Moore. Locally weighted learning. Artificial Intelligence Review, 11:11–73, 1997.

    Article  Google Scholar 

  10. R. J. Aumann. Correlated equilibrium as an expression of Bayesian rationality. Econometrica, 55(1):1–18, 1987.

    Article  MathSciNet  MATH  Google Scholar 

  11. R.J. Aumann and S. Hart. Handbook of Game Theory with Economic Applications, Volumes I and II. North-Holland Press, 1992.

    Google Scholar 

  12. R. Axelrod. The Evolution of Cooperation. Basic Books, New York, 1984.

    Google Scholar 

  13. R. Axelrod. The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton University Press, New Jersey, 1997.

    Google Scholar 

  14. P. Bak and K. Sneppen. Punctuated equilibrium and criticality in a simple model of evolution. Physical Review Letters, 71(24):4083–6, 1993.

    Article  Google Scholar 

  15. P. Bak, C. Tang, and K. Wiesenfeld. Self-organized criticality. Physical Review A, 38: 364, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  16. M. Bando, K. Hasebe, A. Nakayama, A. Shibata, and Y. Sugiyama. Dynamical model of traffic congestion and numerical simulation. Physical Review E, 51(2): 1035–42, 1995.

    Article  Google Scholar 

  17. S. Bankes. Exploring the foundations of artificial societies: Experiments in evolving solutions to the iterated Af-player prisoner's dilemma. In R. Brooks and P. Maes, editors, Artificial Life IV, pages 337–42. MIT Press, 1994.

    Google Scholar 

  18. Y. Bar-Yam, editor. The Dynamics of Complex Systems. Westview Press, 1997.

    Google Scholar 

  19. T. Başar and G. J. Olsder. Dynamic Noncooperative Game Theory, second edition. Siam, Philadelphia, 1999.

    MATH  Google Scholar 

  20. T. Bass. Road to ruin. Discover, 13(5):56–61, May 1992.

    Google Scholar 

  21. M. Batty. Predicting where we walk. Nature, 388:19–20, July 1997.

    Article  Google Scholar 

  22. E. Baum. Toward a model of mind as a laissez-faire economy of idiots. In L. Saitta, editor, Proceedings of the 13th International Conference on Machine Learning, pages 28–36. Morgan Kaufman, 1996.

    Google Scholar 

  23. E. Baum. Toward a model of mind as an economy of agents. Machine Learning, 1999 (in press).

    Google Scholar 

  24. M. Begon, D. J. Thompshon, and M. Mortimer, editors. Population Ecology: A Unified Study of Animals and Plants. Blackwell Science Inc., 1996.

    Google Scholar 

  25. A. M. Bell and W. A. Sethares. The El Farol problem and the internet: Congestion and coordination failure. In Fifth International Conference of the Society for Computational Economics, Boston, 1999.

    Google Scholar 

  26. J. Bendor and P. Swistak. The evolutionary advantage of conditional cooperation. Complexity, 4(2):15–18, 1996.

    Article  Google Scholar 

  27. J. Berg and A. Engel. Matrix games, mixed strategies, and statistical mechanics. Physics Review Letters, 81:4999–5002, 1998. preprint cond-mat/9809265.

    Article  Google Scholar 

  28. D. Bertsekas and R. Gallager. Data Networks. Prentice Hall, Englewood Cliffs, NJ, 1992.

    MATH  Google Scholar 

  29. O. Biham and A. A. Middleton. Self-organization and a dynamical transition in trafficflow models. Physical Review A, 46(10):R6124–7, 1992.

    Article  Google Scholar 

  30. K. Binmore. Fun and Games: A Text on Game Theory. D. C. Heath and Company, Lexington, MA, 1992.

    MATH  Google Scholar 

  31. L. E. Blume and D. Easley. Optimality and natural selection in markets. Preprint: econwpa 9712003.pdf, 1997.

    Google Scholar 

  32. E. Bonabeau, E Henaux, S. Guerin, D. Snyders, P. Kuntz, and G. Theraulaz. Routing in telecommunications networks with “smart” and-like agents (Preprint), 1999.

    Google Scholar 

  33. E. Bonabeau, A. Sobkowski, G. Theraulaz, and J.-L. Deneubourg. Adaptive task allocation inspired by a model of division of labor of social insects (Preprint), 1999.

    Google Scholar 

  34. V. S. Borkar, S. Jain, and G. Rangarajan. Collective behaviour and diversity in economic communities: Some insights from an evolutionary game. In Proceedings of the Workshop on Econophysics, Budapest, Hungary, 1997.

    Google Scholar 

  35. V. S. Borkar, S. Jain, and G. Rangarajan. Dynamics of individual specialization and global diversification in communities. Complexity, 3(3):50–6, 1998.

    Article  Google Scholar 

  36. C. Boutilier. Planning, learning and coordination in multiagent decision processes. In Proceedings of the Sixth Conference on Theoretical Aspects of Rationality and Knowledge, Holland, 1996.

    Google Scholar 

  37. C. Boutilier. Learning conventions in multiagent stochastic domains using likelihood estimates (Preprint), 1999.

    Google Scholar 

  38. C. Boutilier, Y. Shoham, and M. P. Wellman. Editorial: Economic principles of multiagent systems. Artificial Intelligence Journal, 94:1–6, 1997.

    Article  MATH  Google Scholar 

  39. J. A. Boyan and M. Littman. Packet routing in dynamically changing networks: A reinforcement learning approach. In Advances in Neural Information Processing Systems— 6, pages 671–8. Morgan Kaufman, 1994.

    Google Scholar 

  40. J. M. Bradshaw, editor. Software Agents. MIT Press, 1997.

    Google Scholar 

  41. R. A. Brooks. Intelligence without reason. In Proceedings of the Twelfth International Joint Conference on Artificial Intelligence, pages 569–95, 1991.

    Google Scholar 

  42. R. A. Brooks. Intelligence without representation. Artificial Intelligence, 47:139–59, 1991.

    Article  Google Scholar 

  43. T. X. Brown, H. Tong, and S. Singh. Optimizing admission control while ensuring quality of service in multimedia networks via reinforcement learning. In Advances in Neural Information Processing Systems—11. MIT Press, 1999.

    Google Scholar 

  44. G. Caldarelli, M. Marsili, and Y. C. Zhang. A prototype model of stock exchange. Europhysics Letters, 40:479–84, 1997.

    Article  Google Scholar 

  45. A. R. Cassandra, L. P. Kaelbling, and M. L. Littman. Acting optimally in partially observable stochastic domains. In Proceedings of the 12th National Conference on Artificial Intelligence, 1994.

    Google Scholar 

  46. A. Cavagna. Irrelevance of memory in the minority game. Preprint cond-mat/9812215, December 1998.

    Google Scholar 

  47. D. Challet and Y. C. Zhang. Emergence of cooperation and organization in an evolutionary game. Physica A, 246(3-4):407, 1997.

    Article  Google Scholar 

  48. D. Challet and Y. C. Zhang. On the minority game: Analytical and numerical studies. Physica A, 256:514, 1998.

    Article  Google Scholar 

  49. J. Cheng. The mixed strategy equilibria and adaptive dynamics in the bar problem. Technical report, Santa Fe Institute Computational Economics Workshop, 1997.

    Google Scholar 

  50. D. R. Cheriton and K. Harty. A market approach to operating system memory allocation. In S. E. Clearwater, editor, Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientific, 1995.

    Google Scholar 

  51. S. P. M. Choi and D. Y Yeung. Predictive Q-routing: A memory based reinforcement learning approach to adaptive traffic control. In D._S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems—8, pages 945–51. MIT Press, 1996.

    Google Scholar 

  52. D. J. Christini and J. J. Collins. Using noise and chaos control to control nonchaotic systems. Physical Review E, 52(6):5806–9, 1995.

    Article  Google Scholar 

  53. C. Claus and C. Boutilier. The dynamics of reinforcement learning cooperative multiagent systems. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 746–52, Madison, WI, June 1998.

    Google Scholar 

  54. J. E. Cohen and C. Jeffries. Congestion resulting from increased capacity in single-server queueing networks. IEEE/ACM Transactions on Networking, 5(2):305–10, 1997.

    Article  Google Scholar 

  55. J. E. Cohen and F. P. Kelly. A paradox of congestion in a queueing network. Journal of Applied Probability, 27:730–4, 1990.

    Article  MathSciNet  MATH  Google Scholar 

  56. R. H. Crites and A. G. Barto. Improving elevator performance using reinforcement learning. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems—8, pages 1017–23. MIT Press, 1996.

    Google Scholar 

  57. J. de Boer, B. Derrida, H. Flyvberg, A. D. Jackson, and T. Wettig. Simple model of self-organized biological evolution. Physical Review Letters, 73(6):906–9, 1994.

    Article  Google Scholar 

  58. M. A. R. de Cara, O. Pla, and F. Guinea. Competition, efficiency and collective behavior in the “El Farol” Bar model. European Physical Journal B, 10:187, 1999.

    Google Scholar 

  59. A. de Vany. The emergence and evolution of self-organized coalitions. In M. Gilli, editor, Computational Methods in Economics. Kluwer Scientific Publishers, 1999 (to appear).

    Google Scholar 

  60. W. L. Ditto, S. N. Rauseo, and M. L. Spano. Experimental control of chaos. Physics Review Letters, 65:3211, 1990.

    Article  Google Scholar 

  61. W. L. Ditto and K. Showalter. Introduction: Control and synchronization of chaos. Chaos, 1(4):509–11, 1997.

    Article  MathSciNet  Google Scholar 

  62. M. Dorigo and L. M. Gambardella. Ant colonies for the travelling salesman problem. Biosystems, 39, 1997.

    Google Scholar 

  63. M. Dorigo and L. M. Gambardella. Ant colony systems: A cooperative learning approach to the travelling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1):53–66, 1997.

    Article  Google Scholar 

  64. K. E. Drexler. Nanosystems: Molecular Machinery, Manufacturing, and Computation. John Wiley and Sons, 1992.

    Google Scholar 

  65. B. Drossel. A simple model for the formation of a complex organism. Preprint adaporg/9811002, November 1998.

    Google Scholar 

  66. J. Eatwell, M. Milgate, and P. Newman. The New Palgrave Game Theory. Macmillan Press, 1989.

    Google Scholar 

  67. A. A. Economides and J. A. Silvester. Multi-objective routing in integrated services networks: A game theory approach. In IEEE Infocom ′91: Proceedings of the Conference on Computer Communication, volume 3, 1991.

    Google Scholar 

  68. N. Eldredge and S. J. Gould. Punctuated equilibria: An alternative to phyletic gradualism. In J. M. Schopf, editor, Models in Paleobiology, pages 82–115. Greeman, Cooper, 1972.

    Google Scholar 

  69. C. M. Ellison. The Utah TENEX scheduler. Proceedings of the IEEE, 63:940–5, 1975.

    Article  Google Scholar 

  70. J. M. Epstein. Zones of cooperation in demographic prisoner's dilemma. Complexity, 4(2):36–48, 1996.

    Article  Google Scholar 

  71. J. M. Epstein. Nonlinear Dynamics, Mathematical Biology, and Social Science. Addison Wesley, Reading, MA, 1997.

    MATH  Google Scholar 

  72. J. M. Epstein and R. Axtell. Growing Artificial Societies: Social Sciences from the Bottom Up. MIT Press, 1996.

    Google Scholar 

  73. N. Feltovich. Equilibrium and reinforcement learning with private information: An experimental study. Preprint, Dept. of Economics, U. of Houston, July 1997.

    Google Scholar 

  74. J. Ferber. Reactive distributed artificial intelligence: Principles and applications. In G. O'Hare and N. Jennings, editors, Foundations of Distributed Artificial Intelligence, pages 287–314. John Wiley and Sons, 1996.

    Google Scholar 

  75. D. F. Ferguson, C. Nikolaou, and Y. Yemini. An economy for flow control in computer networks. In IEEE Infocom ′89, pages 110–8, 1989.

    Google Scholar 

  76. S. G. Ficici and J. B. Pollack. Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In C. Adami et al., editor, Artificial Life VI, pages 238–47. MIT Press, 1998.

    Google Scholar 

  77. J. Filar and K. Vrieze. Competitive Markov Decision Processes. Springer-Verlag, 1997.

    Google Scholar 

  78. D. B. Fogel. An overview of evolutionary programming. In L. D. Davis, K. De Jong, M. D. Vose, and L. D. Whitley, editors, Evolutionary Algorithms, pages 89–109. Springer, 1997.

    Google Scholar 

  79. C. L. Forgy. RETE: A fast algorithm for the many pattern/many object patent match problem. Artificial Intelligence, 19(1): 17–37, 1982.

    Article  Google Scholar 

  80. D. Freedman. Markov Chains. Springer-Verlag, 1983.

    Google Scholar 

  81. E. Friedman. Strategic properties of heterogeneous serial cost sharing. In Mathematical Social Sciences. 2000.

    Google Scholar 

  82. E. Friedman and D. C. Parkes. Pricing WiFi at Starbucks-Issues in online mechanism design. In Fourth ACM Conf. on Electronic Commerce, 2003.

    Google Scholar 

  83. E. Friedman and S. Shenker. Learning and implementation in the Internet. Available from www.orie.cornell.edu/~friedman, 2002.

  84. J. W. Friedman. Game Theory with Applications to Economics. Oxford University Press, New York, 1986.

    Google Scholar 

  85. D. Fudenberg and D. K. Levine. Steady state learning and Nash equilibrium. Econometrica, 61(3):547–73, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  86. D. Fudenberg and D. K. Levine. The Theory of Learning in Games. MIT Press, 1998.

    Google Scholar 

  87. D. Fudenberg and J. Tirole. Game Theory. MIT Press, 1991.

    Google Scholar 

  88. Gabora. Autocatalytic closure in a cognitive system: A tentative scenario for the origin of culture. Psycoloquy, 9(67), December 1998.

    Google Scholar 

  89. V. V Gafiychuk. Distributed self-regulation induced by negative feedbacks in ecological and economic systems. Preprint, adap-org/98110011, November 1998.

    Google Scholar 

  90. S. Galam. Spontaneous coalition forming: A model from spin glass. Preprint condmat/9901022, January 1999.

    Google Scholar 

  91. C. W. Gardiner. Handbook of Stochastic Methods. Springer-Verlag, New York, 1985.

    Google Scholar 

  92. C. V. Goldman and J. S. Rosenschein. Emergent coordination through the use of cooperative state-changing rules (Preprint), 1999.

    Google Scholar 

  93. S. J. Gould and N. Eldredge. Punctuated equilibria: The tempo and mode of evolution reconsidered. Paleobiology, 3:115–51, 1977.

    Google Scholar 

  94. W. Grover. Self organizing broad band transport networks. Proceedings of the IEEE, 85(10):1582–1611, 1997.

    Article  Google Scholar 

  95. O. Guenther, T. Hogg, and B. A. Huberman. Learning in multiagent control of smart matter. In AAAI-97 Workshop on Multiagent Learning, 1997.

    Google Scholar 

  96. O. Guenther, T. Hogg, and B. A. Huberman. Market organizations for controlling smart matter. In Proceedings of the International Conference on Computer Simulation and Social Sciences, 1997.

    Google Scholar 

  97. E. A. Hansen, A. G. Barto, and S. Zilberstein. Reinforcement learning for mixed openloop and closed loop control. In Advances in Neural Information Processing Systems—9, pages 1026–32. MIT Press, 1998.

    Google Scholar 

  98. I Hanski. Be diverse, be predictable. Nature, 390:440–1, 1997.

    Article  Google Scholar 

  99. A. Hastings. Population Biology: Concepts and Models. Springer-Verlag, 1997.

    Google Scholar 

  100. D. Helbing, J. Keltsch, and P. Molnar. Modeling the evolution of the human trail systems. Nature, 388:47–9, July 1997.

    Article  Google Scholar 

  101. D. Helbing, F. Schweitzer, J. Keltsch, and P. Molnar. Active walker model for the formation of human and animal trail systems. Physical Review E, 56(3):2527–39, 1997.

    Article  Google Scholar 

  102. D. Helbing and M. Treiber. Jams, waves, and clusters. Science, 282:200–1, December 1998.

    Article  Google Scholar 

  103. D. Helbing and M. Treiber. Phase diagram of traffic states in the presence of inhomogeneities. Physics Review Letters, 81:3042, 1998.

    Article  Google Scholar 

  104. M. Herrmann and B. S. Kerner. Local cluster effect in different traffic flow models. PhysicaA, 225:163–8, 1998.

    Article  Google Scholar 

  105. Yu-Chi Ho. Team decision theory and information structures. Proceedings of the IEEE, 68(644-54), 1980.

    Google Scholar 

  106. T. Hogg and B. A. Huberman. Achieving global stability through local controls. In Proceedings of the Sixth IEEE Symposium on Intelligent Control, pages 67–72, 1991.

    Google Scholar 

  107. T. Hogg and B. A. Huberman. Controlling smart matter. Smart Materials and Structures, 7:R1–R14, 1998.

    Article  Google Scholar 

  108. J. Holland and J. H. Miller. Artificial adaptive agents in economic theory. American Economic Review, 81:365–70, May 1991.

    Google Scholar 

  109. J. H. Holland, editor. Adaptation in Natural and Artificial Systems. MIT Press, 1993.

    Google Scholar 

  110. M.-T. T. Hsiao and A. A. Lazar. Optimal flow control of multi-class queueing networks with decentralized information. In IEEE Infocom ′89, pages 652–61, 1987.

    Google Scholar 

  111. J. Hu and M. P. Wellman. Self-fulfilling bias in multiagent learning. In Proceedings of the Second International Conference on Multiagent Systems, pages 118–25, 1996.

    Google Scholar 

  112. J. Hu and M. P. Wellman. Multiagent reinforcement learning: Theoretical framework and an algorithm. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 242–50, June 1998.

    Google Scholar 

  113. J. Hu and M. P. Wellman. Online learning about other agents in a dynamic multiagent system. In Proceedings of the Second International Conference on Autonomous Agents, pages 239–46, May 1998.

    Google Scholar 

  114. M. Huber and R. A. Grupen. Learning to coordinate controllers—Reinforcement learning on a control basis. In Proceedings of the 15th International Conference of Artificial Intelligence, volume 2, pages 1366–71, 1997.

    Google Scholar 

  115. B. A. Huberman, editor. The Ecology of Computation. North-Holland, Amsterdam, 1988.

    Google Scholar 

  116. B. A. Huberman and S. H. Clearwater. A multi-agent system for controlling building environments. In Proceedings of the International Conference on Multiagent Systems, pages 171–6, 1995.

    Google Scholar 

  117. B. A. Huberman and T. Hogg. The behavior of computational ecologies. In The Ecology of Computation, pages 77–115. North-Holland, 1988.

    Google Scholar 

  118. M. E. Huhns, editor. Distributed Artificial Intelligence. Pittman, London, 1987.

    Google Scholar 

  119. R. V. Iyer and S. Ghosh. DARYN, a distributed decision-making algorithm for railway networks: Modeling and simulation. IEEE Transaction of Vehicular Technology, 44(1):180–91, 1995.

    Article  Google Scholar 

  120. P. Jefferies, M. L. Hart, and N. F. Johnson. Deterministic dynamics in the minority game. Physical Review E, 65 (016105), 2002.

    Article  Google Scholar 

  121. N. R. Jennings, K. Sycara, and M. Wooldridge. A roadmap of agent research and development. Autonomous Agents and Multi-agent Systems, 1:7–38, 1998.

    Article  Google Scholar 

  122. N. F. Johnson, S. Jarvis, R. Jonson, P. Cheung, Y. R. Kwong, and P. M. Hui. Volatility and agent adaptability in a self-organizing market. Preprint cond-mat/9802177, February 1998.

    Google Scholar 

  123. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–85, 1996.

    Google Scholar 

  124. E. Kalai and E. Lehrer. Rational learning leads to Nash equilibrium. Econometrica, 61(5): 1019–45, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  125. S.A. Kauffman. At Home in the Universe: The Search for the Laws of Self-Organization and Complexity. Oxford University Press, 1995.

    Google Scholar 

  126. L. Keller and H. K. Reeve. Familiarity breeds cooperation. Nature, 394:121–2, 1998.

    Article  Google Scholar 

  127. F. P. Kelly. Modeling communication networks, present and future. Philosophical Trends Royal Society of London A, 354:437–63, 1996.

    Article  MATH  Google Scholar 

  128. J. O. Kephart, J. E. Hanson, and J. Sairamesh. Price and niche wars in a free-market economy of software agents. Artificial Life, 4:1–13, 1998.

    Article  Google Scholar 

  129. B. S. Kerner, P. Konhauser, and M. Schilke. Deterministic spontaneous appearance of traffic jams in slightly inhomogeneous traffic flow. Physical Review E, 51(6):6243–6, 1995.

    Article  Google Scholar 

  130. B. S. Kerner and H. Rehborn.Experimental properties of complexity in traffic flow. Physical Review E, 53(5):R4275–8, 1996.

    Article  Google Scholar 

  131. T. F. Knight and G. J. Sussman. Cellular gate technology. In Proceedings of the First International Conference on Unconventional Models of Computation, Auckland, January 1998.

    Google Scholar 

  132. Y. A. Korilis, A. A. Lazar, and A. Orda. Achieving network optima using Stackelberg routing strategies. IEEE/ACM Transactions on Networking, 5(1): 161–73, 1997.

    Article  Google Scholar 

  133. Y. A. Korilis, A. A. Lazar, and A. Orda. Capacity allocation under noncooperative routing. IEEE Transactions on Automatic Control, 42(3):309–25, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  134. Y. A. Korilis, A. A. Lazar, and A. Orda. Avoiding the Braess paradox in noncooperative networks. Journal of Applied Probability, 36:211–22, 1999.

    Article  MathSciNet  MATH  Google Scholar 

  135. S. Kraus. Negotiation and cooperation in multi-agent environments. Artificial Intelligence, pages 79–97, 1997.

    Google Scholar 

  136. M. J. B. Krieger, J.-B. Billeter, and L. Keller. Ant-like task allocation and recruitment in cooperative robots. Nature, 406:992–5, 2000.

    Article  Google Scholar 

  137. V. Krishna and P. Motty. Efficient mechanism design. (Preprint), 1997.

    Google Scholar 

  138. J. F. Kurose and R. Simha. A microeconomic approach to optimal resource allocation in distributed computer systems. IEEE Transactions on Computers, 35(5):705–17, 1989.

    Article  Google Scholar 

  139. R. J. La and V. Anantharam. Optimal routing control: Game theoretic approach (Submitted to IEEE transactions on Automatic Control), 1999.

    Google Scholar 

  140. M. Lauer and M. Riedmiller. An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In Proceedings of the Seventeenth International Machine Learning Conference, pages 535–42. Morgan Kauffman, 2000.

    Google Scholar 

  141. A. A. Lazar, A. Orda, and D. E. Pendarakis. Capacity allocation under noncooperative routing. IEEE Transactions on Networking, 5(6):861–71, 1997.

    Article  Google Scholar 

  142. A. A. Lazar and N. Semret. Design, analysis and simulation of the progressive second price auction for network bandwidth sharing. Technical Report 487-98-21 (Rev 2.10), Columbia University, April 1998.

    Google Scholar 

  143. T. S. Lee, S. Ghosh, J. Liu, X. Ge, and A. Nerode. A mathematical framework for asynchronous, distributed, decision-making systems with semi-autonomous entities: Algorithm synthesis, simulation, and evaluation. In Fourth International Symposium on Autonomous Decentralized Systems, Tokyo, 1999.

    Google Scholar 

  144. T. M. Lenton. Gaia and natural selection. Nature, 394:439–447, 1998.

    Article  Google Scholar 

  145. K. Lerman and A. Galstyan. Mathematical model of foraging in a group of robots: Effect of interference. Autonomous Robots, 13(2), 2002.

    Google Scholar 

  146. M. L. Littman. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the 11th International Conference on Machine Learning, pages 157–63, 1994.

    Google Scholar 

  147. M. L. Littman and J. Boyan. A distributed reinforcement learning scheme for network routing. In Proceedings of the 1993 International Workshop on Applications of Neural Networks to Telecommunications, pages 45–51, 1993.

    Google Scholar 

  148. R. D. Luce and H. Raiffa. Games and Decisions. Dover Press, 1985.

    Google Scholar 

  149. J. K. MacKie-Mason and R. V. Hal. Pricing congestible network resources. IEEE Journal on Selected Areas of Communications, 13(7):1141–49, 1995.

    Article  Google Scholar 

  150. W. G. Macready and D. H. Wolpert. Bandit problems and the exploration/exploitation tradeoff. IEEE Transactions on Evolutionary Computation, 2:2–22, 1998.

    Article  Google Scholar 

  151. P.Maes. Designing Autonomous Agents. MIT Press, 1990.

    Google Scholar 

  152. P. Marbach, O. Mihatsch, M. Schulte, and J. Tsisiklis. Reinforcement learning for call admission control and routing in integrated service networks. In Advances in Neural Information Processing Systems—10, pages 922–8. MIT Press, 1998.

    Google Scholar 

  153. J. Marschak and R. Radner. Economic Theory of Teams. Yale University Press, New Haven, CT, 1972.

    MATH  Google Scholar 

  154. M. Marsili and Y.-C. Zhang. Stochastic dynamics in game theory. Preprint condmat/9801309, January 1998.

    Google Scholar 

  155. J. Maynard Smith. Evolution and the Theory of Games. Cambridge University Press, 1982.

    Google Scholar 

  156. D. McFarland. Toward robot cooperation. In From Animals to Animais 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior, pages 440–3. MIT Press, 1994.

    Google Scholar 

  157. M. Mesterton-Gibbons, J. H. Marden, and L. A. Dugatkin. On wars of attrition without assessment. Journal of Theoretical Biology, 181:65–83, 1992.

    Article  Google Scholar 

  158. D. A. Meyer and T. A. Brown. Statistical mechanics of voting. Physics Review Letters, 81(8):1718–21, 1998.

    Article  Google Scholar 

  159. J. H. Miller. The coevolution of automata in the repeated prisoner's dilemma. Journal of Economic Behavior and Organization, 29(1):87–112, 1996.

    Article  Google Scholar 

  160. J. H. Miller. Evolving information processing organizations (Preprint), 1996.

    Google Scholar 

  161. J. H. Miller and J. Andreoni. Auctions with adaptive artificial agents. Journal of Games and Economic Behavior, 10:39–64, 1995.

    Article  MATH  Google Scholar 

  162. J. H. Miller, C. Butts, and D. Rode. Communication and cooperation (Preprint), 1998.

    Google Scholar 

  163. M. Minsky. The Society of Mind. Simon and Schuster, 1988.

    Google Scholar 

  164. J. Mirrlees. An exploration in the theory of optimal income taxation. Review of Economic Studies, 38:175–208, 1974.

    Article  Google Scholar 

  165. A. W. Moore, C. G. Atkeson, and S. Schaal. Locally weighted learning for control. Artificial Intelligence Review, 11:75–113, 1997.

    Article  Google Scholar 

  166. R. Munos and P. Bourgine. Reinforcement learning for continuous stochastic control problems. In Advances in Neural Information Processing Systems—10, pages 1029–35. MIT Press, 1998.

    Google Scholar 

  167. R. Nagpal. Programmable pattern-formation and scale-independence. In Proceedings of the 4th International Conference on Complex Systems, New Hampshire, June 2002.

    Google Scholar 

  168. R. Nagpal. Programmable self-assembly using biologically-inspired multi-agent control. In Proceedings of the 1st International Joint Conference on Autonomous Agents and Multi-agent Systems, July 2002.

    Google Scholar 

  169. K. Naigel. Experiences with iterated traffic microsimulations in Dallas. Preprint adaporg/9712001, December 1997.

    Google Scholar 

  170. K. Naigel, P. Stretz, M. Pieck, S. Leckey, R. Donnelly, and C. Barrett. TRANSIMS traffic flow characteristics. Preprint adap-org/9710003, October 1997.

    Google Scholar 

  171. J. F. Nash. Equilibrium points in TV-person games. Proceedings of the National Academy of Sciences of the United States of America, 36(48-49), 1950.

    Google Scholar 

  172. R. M. Neal. Bayesian Learning for Neural Networks, Lecture Notes in Statistics, No. 118. Springer-Verlag, New York, 1996.

    Book  Google Scholar 

  173. A. Neyman. Bounded complexity justifies cooperation in the finitely repeated prisoner's dilemma. Economics Letters, 19:227–30, 1985.

    Article  MathSciNet  Google Scholar 

  174. W. Nicholson. Microeconomic Theory, seventh edition. The Dryden Press, 1998.

    Google Scholar 

  175. N. Nisan and A. Ronen. Algorithmic mechanism design. Games and Economic Behavior, 35:166–96, 2001.

    Article  MathSciNet  MATH  Google Scholar 

  176. S. I. Nishimura and T. Ikegami. Emergence of collective strategies in a prey-predator game model. Artificial Life, 3:243–360, 1997.

    Article  Google Scholar 

  177. J. Norris. Markov Chains. Cambridge University Press, 1998.

    Google Scholar 

  178. M. A. Nowak and K. Sigmund. Evolution of indirect reciprocity by image scoring. Nature, 393:573–7, 1998.

    Article  Google Scholar 

  179. S. Olafsson. Games on networks. Proceedings of the IEEE, 85(10): 1556–62, 1997.

    Article  Google Scholar 

  180. A. Orda, R. Rom, and M. Sidi. Minimum delay routing in stochastic networks. IEEE/ACM Transactions on Networking, 1(2): 187–98, 1993.

    Article  Google Scholar 

  181. D. C. Parkes. Iterative Combinatorial Auctions: Theory and Practice. Ph.D. thesis, University of Pennsylvania, 2001.

    Google Scholar 

  182. D. C. Parkes. Price-based information certificates for minimal-revelation combinatorial auctions. In Agent Mediated Electronic Commerce IV: Designing Mechanisms and Systems, volume 2531 of Lecture Notes in Artificial Intelligence. 2002.

    Google Scholar 

  183. L. A. Pipes. An operational analysis of traffic dynamics. Journal of Applied Physics, 24(3):274–81, 1953.

    Article  MathSciNet  Google Scholar 

  184. G. A. Polls. Stability is woven by complex webs. Nature, 395:744–5, 1998.

    Article  Google Scholar 

  185. M. Potters, R. Cont, and J.-P. Bouchaud. Financial markets as adaptive ecosystems. Preprint cond-mat/9609172 v2, June 1997.

    Google Scholar 

  186. D. Prokhorov and D. Wunsch. Adaptive critic design. IEEE Transactions on Neural Networks, 8(5):997–1007, 1997.

    Article  Google Scholar 

  187. Z. Qu, F. Xie, and G. Hu. Spatiotemporal on-off intermittency by random driving. Physical Review E, 53(2):R1301–4, 1996.

    Article  Google Scholar 

  188. F. Reif. Fundamentals of Statistical and Thermal Physics. McGraw-Hill, 1965.

    Google Scholar 

  189. E. Rich and K. Knight. Artificial Intelligence, second edition. McGraw-Hill, Inc., 1991.

    Google Scholar 

  190. A. E. Roth and I. Erev. Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8:164–212, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  191. A. Samuel. Some studies in machine learning using the game of checkers. IBM Journal of Reseach and Development, 3:210–29, 1959.

    Article  Google Scholar 

  192. T. Sandholm and R. Crites. Multiagent reinforcement learning in the iterated prisoner's dilemma. Biosystems, 37:147–66, 1995.

    Article  Google Scholar 

  193. T. Sandholm, K. Larson, M. Anderson, O. Shehory, and F. Tohme. Anytime coalition structure generation with worst case guarantees. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 46–53, 1998.

    Google Scholar 

  194. T. Sandholm and V. R. Lesser. Issues in automated negotiations and electronic commerce: Extending the contract net protocol. In Proceedings of the Second International Conference on Multi-agent Systems, pages 328–35. AAAI Press, 1995.

    Google Scholar 

  195. T. Sandholm and V. R. Lesser. Coalitions among computationally bounded agents. Artificial Intelligence, 94:99–137, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  196. S. Sastry and M. Bodson. Adaptive Control, Stability, Convergence, and Robustness. Prentice Hall, 1989.

    Google Scholar 

  197. R. Savit, R. Manuca, and R. Riolo. Adaptive competition, market efficiency, phase transitions and spin-glasses. Preprint cond-mat/9712006, December 1997.

    Google Scholar 

  198. A. Schaerf, Y. Shoham, and M. Tennenholtz. Adaptive load balancing: A study in multiagent learning. Journal of Artificial Intelligence Research, 162:475–500, 1995.

    Google Scholar 

  199. J. Schmidhuber, J. Zhao, and N. N. Schraudoiph. Reinforcement learning with selfmodifying policies. In S. Thrun and L. Pratt, editors, Learning to Learn, pages 293–309. Kluwer, 1997.

    Google Scholar 

  200. J. Schmidhuber, J. Zhao, and M. Wiering. Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28:105–30, 1997.

    Article  Google Scholar 

  201. R. Schoonderwoerd, O. Holland, and J. Bruten. Ant-like agents for load balancing in telecommunication networks. In Autonomous Agents 97, pages 209–16. MIT Press, 1997.

    Google Scholar 

  202. M. Schreckenberg, A. Schadschneider, K. Nagel, and N. Ito. Discrete stochastic models for traffic flow. Physical Review E, 51(4):2939–49, 1995.

    Article  Google Scholar 

  203. J. Schull. Are species intelligent? Behavioral and Brain Sciences, 13:63–108, 1990.

    Article  Google Scholar 

  204. S. Sen. Multi-agent Learning: Papers from the 1997 AAAI Workshop (Technical Report WS-97-03. AAAI Press, Menlo Park, CA, 1997.

    Google Scholar 

  205. S. Sen, M. Sekaran, and J. Hale. Learning to coordinate without sharing information (Preprint), 1999.

    Google Scholar 

  206. W. A. Sethares and A. M. Bell. An adaptive solution to the El Farol problem. In Proceedings, of the Thirty-Sixth Annual Allerton Conference on Communication, Control, and Computing, Allerton, IL, 1998.

    Google Scholar 

  207. R. Sethi. Stability of equilibria in games with procedural rational players. Preprint, Dept of Economics, Columbia University, November 1998.

    Google Scholar 

  208. S. J. Shenker. Making greed work in networks: A game-theoretic analysis of switch service disciplines. IEEE Transactions on Networking, 3(6):819–31, 1995.

    Article  MathSciNet  Google Scholar 

  209. Y. Shoham and K. Tanaka. A dynamic theory of incentives in multi-agent systems. In Proceedings of the International Joint Conference on Artificial Intelligence, 1997.

    Google Scholar 

  210. J. Sidel, P. M. Aoki, S. Barr, A. Sah, C. Staelin, M. Stonebreaker, and Yu A. Data replication in mariposa. In Proceedings of the 12th International Conference on Data Engineering, 1996.

    Google Scholar 

  211. S. Sinha and N. Gupte. Adaptive control of spatially extended systems: Targeting spatiotemporal patterns and chaos. Physical Review E, 58(5):R5221–4, 1998.

    Article  Google Scholar 

  212. W. Stallings. Data and Computer Communications. MacMillian Publishing Co., New York, 1994.

    MATH  Google Scholar 

  213. J. Stein. Critical exponents of the u(n) vector spin glasses. Europhysics Letters, 34(9):717–21, 1996.

    Article  Google Scholar 

  214. J. Stein. Critical properties of a spin glass with anisotropic Dzyaloshinskii-Moriya interaction. Journal of Physics A, 29:963–71, 1996.

    Article  MATH  Google Scholar 

  215. W. J. Stewart. Introduction to the Numerical Solution of Markov Chains. Princeton University Press, 1995.

    Google Scholar 

  216. P. Stone. TPOT-RL applied to network routing. In Proceedings of the Seventeenth International Machine Learning Conference, pages 935–42. Morgan Kauffman, 2000.

    Google Scholar 

  217. P. Stone and M. Veloso. Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3), 2000.

    Google Scholar 

  218. M. Stonebreaker, P. M. Aoki, R. Devine, W. Litwin, and M. Olson. Mariposa: A new architecture for distributed data. In Proceedings of the 10th International Conference on Data Engineering, 1994.

    Google Scholar 

  219. D. Subramanian, P. Druschel, and J. Chen. Ants and reinforcement learning: A case study in routing in dynamic networks. In Proceedings of the Fifteenth International Conference on Artificial Intelligence, pages 832–8, 1997.

    Google Scholar 

  220. R. S. Sutton. Learning to predict by the methods of temporal differences. Machine Learning, 3:9–44, 1988.

    Google Scholar 

  221. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.

    Google Scholar 

  222. K. Sycara. Multiagent systems. AI Magazine, 19(2):79–92, 1998.

    Google Scholar 

  223. G. Szabo and C. Toke. Evolutionary prisoner's dilemma game on a square lattice. Physical Review E, 58(1):69–73, 1998.

    Article  Google Scholar 

  224. G. Tesauro. Practical issues in temporal difference learning. Machine Learning, 8:33–53, 1992.

    Google Scholar 

  225. P. Tucker and F. Berman. On market mechanisms as a software techniques. Technical Report CS96-513, University of California, San Diego, December 1996.

    Google Scholar 

  226. K. Turner, A. Agogino, and D. Wolpert. Learning sequences of actions in collectives of autonomous agents. In Proceedings of the First International Joint Conference on Autonomous Agents and Multi-agent Systems, pages 378–85, Bologna, Italy, July 2002.

    Google Scholar 

  227. K. Turner and J. Lawson. Collectives for multiple resource job scheduling across heterogeneous servers. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-agent Systems, Melbourne, Australia, July 2003.

    Google Scholar 

  228. K. Turner and D. H. Wolpert. Collective intelligence and Braess' paradox. In Proceedings of the Seventeenth National Conference on Artificial Intelligence, pages 104–9, Austin, TX, 2000.

    Google Scholar 

  229. W. Vickrey. Counterspeculation, auctions and competitive sealed tenders. Journal of Finance, 16:8–37, 1961.

    Article  Google Scholar 

  230. C. A. Waldspurger, T. Hogg, B. A. Huberman, J. O. Kephart, and W. S. Stornetta. Spawn: A distributed computational economy. IEEE Transactions of Software Engineering, 18(2):103–17, 1992.

    Article  Google Scholar 

  231. J. Walrand and P. Varaiya. High-Performance Communication Networks. Morgan Kaufmann, San Fransisco, 1996.

    Google Scholar 

  232. C. Watkins and P. Dayan. Q-learning. Machine Learning, 8(3/4):279–92, 1992.

    Article  MATH  Google Scholar 

  233. R. Weiss, G. Homsy, and R. Nagpal. Programming biological cells. In Proceedings of the 8th International Conference on Architectural Support for Programming Languages and Operating Systems, San Jose, NZ, 1998.

    Google Scholar 

  234. M. P. Wellman. A market-oriented programming environment and its application to distributed multicommodity flow problems. In Journal of Artificial Intelligence Research, 1993.

    Google Scholar 

  235. M. P. Wellman. A computational market model for distributed configuration design. In Proceedings of the 12th National Conference on Artificial Intelligence, 1994.

    Google Scholar 

  236. D. H. Wolpert and W. G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1):67–82, 1997. Best Paper Award.

    Article  Google Scholar 

  237. D. H. Wolpert, J. Sill, and K. Turner. Reinforcement learning in distributed domains: Beyond team games. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pages 819–24, Seattle, 2001.

    Google Scholar 

  238. D. H. Wolpert and K. Turner. Optimal payoff functions for members of collectives. Advances in Complex Systems, 4(2/3):265–79, 2001.

    Article  MATH  Google Scholar 

  239. D. H. Wolpert and K. Turner. Collective intelligence, data routing and Braess' paradox. Journal of Artificial Intelligence Research, 16:359–87, 2002.

    Article  MathSciNet  MATH  Google Scholar 

  240. D. H. Wolpert, K. Turner, and E. Bandari. Improving search algorithms by using intelligent coordinates. 2003, submitted.

    Google Scholar 

  241. D. H. Wolpert, K. Turner, and J. Frank. Using collective intelligence to route Internet traffic. In Advances in Neural Information Processing Systems—11, pages 952–8. MIT Press, 1999.

    Google Scholar 

  242. D. H. Wolpert, K. Wheeler, and K. Turner. General principles of learning-based multiagent systems. In Proceedings of the Third International Conference of Autonomous Agents, pages 77–83, 1999.

    Google Scholar 

  243. D. H. Wolpert, K. Wheeler, and K. Turner. Collective intelligence for control of distributed dynamical systems. Europhysics Letters, 49(6), March 2000.

    Google Scholar 

  244. S. Wright. The roles of mutation, inbreeding, crossbreeding and selection in evolution. Proceedings of the XI International Congress of Genetics, 8:209–22, 1932.

    Google Scholar 

  245. H. P. Young. The evolution of conventions. Econometrica, 61(1):57–84, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  246. E. Zambrano. Rationalizable bounded rational behavior (Preprint), 1999.

    Google Scholar 

  247. W. Zhang and T. G. Dietterich. Solving combinatorial optimization tasks by reinforcement learning: A general methodology applied to resource-constrained scheduling. Journal of Artificial Intelligence Research, 2000.

    Google Scholar 

  248. Y. C. Zhang. Modeling market mechanism with evolutionary games. Europhysics Letters, March/April 1998.

    Google Scholar 

  249. G. Zlotkin and J. S. Rosenschein. Coalition, cryptography, and stability: Mechanisms for coalition formation in task oriented domains (Preprint), 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NASA Ames Research Center, Mailstop 269-4, Moffett Field, CA, 94035, USA

    Kagan Tumer & David Wolpert

Authors
  1. Kagan Tumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Wolpert
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computational Sciences Division, NASA Ames Research Center, Moffett Field, CA, 94035, USA

    Kagan Tumer  & David Wolpert  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer Science+Business Media New York

About this chapter

Cite this chapter

Tumer, K., Wolpert, D. (2004). A Survey of Collectives. In: Tumer, K., Wolpert, D. (eds) Collectives and the Design of Complex Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8909-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-8909-3_1

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4612-6472-9

  • Online ISBN: 978-1-4419-8909-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation