Evolution of Cooperation within a Behavior-Based Perspective: Confronting Nature and Animats

  • Samuel Delepoulle
  • Philippe Preux
  • Jean-Claude Darcheville
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1829)


We study the evolution of social behaviors within a behavioral framework. To this end, we define a “minimal social situation” that is experimented with both humans and simulations based on reinforcement learning algorithms. We analyse the dynamics of behaviors in this situation by way of operant conditioning. We show that the best reinforcement algorithm, based on Staddon-Zhang’s equations, has a performance and a variety of behaviors that comes close to that of humans, and clearly outperforms the well-known Q-learning. Though we use here a rather simple, yet rich, situation, we argue that operant conditioning deserves much study in the realm of artificial life, being too often misunderstood, and confused with classical conditioning.


Social Situation Operant Conditioning Cooperative Rate Selectionist Approach Reinforcement Learning Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Attonaty, J.M., Chatelin, M.H., Garcia, F., Ndiaye, S.M.: Using extended machine learning and simulation technics to design crop management stategies. In: EFITA First European Conference for Information Technology in Agriculture (1997); CopenhagenGoogle Scholar
  2. 2.
    Axelrod, R.: The evolution of cooperation. Basic Book Inc., New York (1984)Google Scholar
  3. 3.
    Bergen, D.E., Hahn, J.K., Bock, P.: An adaptive approch for reactive actor design. In: Proc. European Conference on Artificial Life (1997)Google Scholar
  4. 4.
    Boren, J.J.: An experimental social relation between two monkeys. Journal of the experimental analysis of behavior 9, 691–700 (1966)CrossRefGoogle Scholar
  5. 5.
    Touretzky, D.S., Saksida, L.M.: Skinnerbots. In: Maes, P., Mataric, M., Meyer, J.-A., Pollack, J., Wilson, S.W. (eds.) Proc. 4th Int’l Conf. on Simulation of Adaptive Behavior, From Animals to Animats 4. MIT Press, Cambridge (1996)Google Scholar
  6. 6.
    Dawkins, R.: The Selfish Gene. Oxford University Press, Oxford (1976)Google Scholar
  7. 7.
    Delepoulle, S., Preux, P., Darcheville, J.C.: Partage des tâches et apprentissage par renforcement. In: Proc. Journées Francophones d’Apprentissage, pp. 201–204 (1998) (in french)Google Scholar
  8. 8.
    Deneubourg, J.L., Goss, S.: Collective paterns and decision-making. Ethology Ecology and Evolution 1, 295–311 (1989)CrossRefGoogle Scholar
  9. 9.
    Donahoe, J.W., Burgos, J.E., Palmer, D.C.: A selectionist approach to rein- forcement. Journal of the experimental analysis of behavior 60, 17–40 (1993)CrossRefGoogle Scholar
  10. 10.
    Hake, D.F., Vukelich, R.: Analysis of the control exerted by a complex cooperation procedure. Journal of the experimental analysis of behavior 19, 3–16 (1973)CrossRefGoogle Scholar
  11. 11.
    Hamilton, W.D.: The Genetical Evolution of Social Behaviour. Journal of Theoritical Biology 7, 1–52 (1964)CrossRefGoogle Scholar
  12. 12.
    Hemelrijk, C.K.: Cooperation without genes, games or cognition. In: Proc. European Conference on Artificial Life (1997)Google Scholar
  13. 13.
    Hilgard, E.R., Bower, G.H.: Theories of learning, 4th edn. Prentice- Hall, Enblewood CliffsGoogle Scholar
  14. 14.
    Hutchinson, W.R.: Teaching an agent to speak and listen with understanding: Why and how? In: Proceedings of the Intelligent Information Agents Workshop, CIKM, Baltimore,
  15. 15.
    Hutchinson, W.R.: The 7G operant behavior toolkit: Software and documentation. Behavior System, Boulder, COGoogle Scholar
  16. 16.
    Ito, A.: How do selfish agents learn to cooperate? In: Langton, C.G., Shimohara, K. (eds.) Proc. Artificial life 5, pp. 185–192. MIT Press, Cambridge (1996)Google Scholar
  17. 17.
    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
  18. 18.
    Langton, C.: Artificial life. In: Langton, R.E. (ed.) Proc. Artificial Life, pp. 1–47. Addison-Wesley, Reading (1987)Google Scholar
  19. 19.
    McFarland, D.: Towards Robot Cooperation. In: Proc of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats 3, pp. 440–444 (1994)Google Scholar
  20. 20.
    Murciano, A., Millán, J.R.: Learning signaling behaviors and specialization in cooperative agents. Adaptive Behavior 5(1), 5–28 (1997)CrossRefGoogle Scholar
  21. 21.
    Sidowski, J.B.: Reward and Punishment in a Minimal Social Situation. Journal of Experimental Psychology 55, 318–326 (1957)CrossRefGoogle Scholar
  22. 22.
    Sidowski, J.B., Wyckoff, B., Tabory, L.: The influence of reinforcement and punishment in a minimal socila situation. Journal of Abnormal Social Psychology 52, 115–119 (1956)CrossRefGoogle Scholar
  23. 23.
    Skinner, B.F.: The behavior of organisms. Prentice Hall, Englewood Cliffs (1938)Google Scholar
  24. 24.
    Skinner, B.F.: Science and human behavior. Macmillan, New York (1953)Google Scholar
  25. 25.
    Skinner, B.F.: Selection by consequence. Science 213, 501–514 (1981)CrossRefGoogle Scholar
  26. 26.
    Staddon, J.E.R.: Adaptive Behavior and Learning. Cambridge University Press, Cambridge (1981)Google Scholar
  27. 27.
    Staddon, J.E.R., Zhang, Y.: On the Assignment-of-Credit Problem in Operant Learning. In: Caumais, M.L., Grossberg, S. (eds.) Neural Network model of Conditioning and Action, Laurence Erlbaum, Hillsdale (1991)Google Scholar
  28. 28.
    Sutton, R.S., Barto, A.G.: Time-Derivative Models of Pavlovian Reinforcement. In: Gabriel, M., Moore, J. (eds.) Learning and Computational Neuroscience: Foundations of Adaptive Networks, pp. 497–537. MIT Press, Cambridge (1990), Google Scholar
  29. 29.
    Sutton, R.S.: Reinforcement Learning. MIT Press, Cambridge (1998)Google Scholar
  30. 30.
    Theraulaz, G., Pratte, M., Gervet, J.: Behavioural profiles in Polistes dominulus (Christ) wasp societies: a quantitative study. Behaviour 113, 223–250 (1990)CrossRefGoogle Scholar
  31. 31.
    Thorndike, E.L.: Animal Intelligence: An experimental study of the associative process in animals. Psychology Monographs 2 (1898)Google Scholar
  32. 32.
    Thorndike, E.L.: Animal Intelligence: Experimental studies. MacMillan, New York (1911)Google Scholar
  33. 33.
    Watkins, C.J.C.H., Dayan, P.: Q-Learning. Technical Note. Machine Learning 8(3), 279–292 (1992)zbMATHGoogle Scholar
  34. 34.
    Wilson, E.O.: The Insect Societies. The Belknap Press, Harvard University Press (1971)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Samuel Delepoulle
    • 1
    • 2
  • Philippe Preux
    • 2
  • Jean-Claude Darcheville
    • 1
  1. 1.UPRES-EA 1059: Unité de Recherche sur l’Evolution du Comportement et des ApprentissagesUniversité de Lille 3Villeneuve d’Ascq CedexFrance
  2. 2.Laboratoire d’Informatique du LittoralCalaisFrance

Personalised recommendations