Advertisement

Animal Cognition

, Volume 18, Issue 5, pp 1093–1103 | Cite as

The learning of action sequences through social transmission

  • Andrew Whalen
  • Daniel CowndenEmail author
  • Kevin Laland
Original Paper

Abstract

Previous empirical work on animal social learning has found that many species lack the ability to learn entire action sequences solely through reliance on social information. Conversely, acquiring action sequences through asocial learning can be difficult due to the large number of potential sequences arising from even a small number of base actions. In spite of this, several studies report that some primates use action sequences in the wild. We investigate how social information can be integrated with asocial learning to facilitate the learning of action sequences. We formalize this problem by examining how learners using temporal difference learning, a widely applicable model of reinforcement learning, can combine social cues with their own experiences to acquire action sequences. The learning problem is modeled as a Markov decision process. The learning of nettle processing by mountain gorillas serves as a focal example. Through simulations, we find that the social facilitation of component actions can combine with individual learning to facilitate the acquisition of action sequences. Our analysis illustrates that how even simple forms of social learning, combined with asocial learning, generate substantially faster learning of action sequences compared to asocial processes alone, and that the benefits of social information increase with the length of the action sequence and the number of base actions.

Keywords

Social learning Sequence learning Temporal difference learning Markov decision process Chaining 

Notes

Acknowledgments

The authors thank Richard Byrne for his very helpful comments. DC thanks Magnus Enquist and Johan Lind for countless inspiring conversations and in particular for pointing out the importance of sequence learning, and the potential of Markov decision processes for modeling such learning problems. This research was supported by a grant from The John Templeton Foundation.

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical standard

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. Berg ME, Grace RC (2006) Initial-link duration and acquisition of preference in concurrent chains. Learn Behav 34(1):50–60CrossRefPubMedGoogle Scholar
  2. Biro D, Inoue-Nakamura N, Tonooka R, Yamakoshi G, Sousa C, Matsuzawa T (2003) Cultural innovation and transmission of tool use in wild chimpanzees: evidence from field experiments. Anim Cogn 6(4):213–223CrossRefPubMedGoogle Scholar
  3. Boesch C, Boesch H (1982) Optimisation of nut-cracking with natural hammers by wild chimpanzees. Behaviour 88(3):265–286Google Scholar
  4. Boyd R, Richerson PJ (1985) Culture and the evolutionary process. University of Chicago Press, ChicagoGoogle Scholar
  5. Brass M, Heyes C (2005) Imitation: is cognitive neuroscience solving the correspondence problem? Trends Cogn Sci 9(10):489–495CrossRefPubMedGoogle Scholar
  6. Buhmann MD (2000) Radial basis functions. Acta Numer 2000(9):1–38CrossRefGoogle Scholar
  7. Bush R, Mosteller F (1951) A mathematical model for simple learning. Psychol Rev 58(5):313CrossRefPubMedGoogle Scholar
  8. Byrne RW (1999) Cognition in great ape ecology. Skill-learning ability opens up foraging opportunities. Symp Zool Soc Lond 72:333–350Google Scholar
  9. Byrne RW (2003) Imitation as behaviour parsing. Philos Trans R Soc B 358(1431):529–536CrossRefGoogle Scholar
  10. Byrne RW, Byrne JM (1993) Complex leaf-gathering skills of mountain gorillas (Gorilla g. beringei): variability and standardization. Am J Primatol 31(4):241–261CrossRefGoogle Scholar
  11. Byrne RW, Russon AE (1998) Learning by imitation: a hierarchical approach. Behav Brain Sci 21(5):667–684PubMedGoogle Scholar
  12. Call J, Tomasello M (1995) Use of social information in the problem solving of orangutans (Pongo pygmaeus) and human children (Homo sapiens). J Comp Psychol 109(3):308CrossRefPubMedGoogle Scholar
  13. Custance D, Whiten A, Fredman T (1999) Social learning of an artificial fruit task in capuchin monkeys (Cebus apella). J Comp Psychol 113(1):13CrossRefGoogle Scholar
  14. Custance D, Whiten A, Sambrook T, Galdikas B (2001) Testing for social learning in the” artificial fruit” processing of wildborn orangutans (Pongo pygmaeus), Tanjung Puting, Indonesia. Anim Cogn 4(3–4):305–313CrossRefPubMedGoogle Scholar
  15. Dayan P, Niv Y (2008) Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol 18(2):185–196 CrossRefPubMedGoogle Scholar
  16. Enquist M, Eriksson K, Ghirlanda S (2007) Critical social learning: a solution to Rogers's paradox of nonadaptive culture. Am Anthropol 109(4):727–734Google Scholar
  17. Eriksson K, Enquist M, Ghirlanda S (2007) Critical points in current theory of conformist social learning. J Evol Psychol 5(1):67–87CrossRefGoogle Scholar
  18. Fantino E, Preston RA, Dunn R (1993) Delay reduction: current status. J Exp Anal Behav 60(1):159–169CrossRefPubMedCentralPubMedGoogle Scholar
  19. Glimcher PW (2011) Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci USA 108(15):647–654Google Scholar
  20. Goodall J (1964) Tool-using and aimed throwing in a community of free-living chimpanzees. Nature 201:1264CrossRefPubMedGoogle Scholar
  21. Grace RC (1994) A contextual model of concurrent-chains choice. J Exp Anal Behav 61(1):113–129CrossRefPubMedCentralPubMedGoogle Scholar
  22. Henrich J, Boyd R (1998) The evolution of conformist transmission and the emergence of between-group differences. Evol Hum Behav 19(4):215–241CrossRefGoogle Scholar
  23. Heyes CM (1994) Social learning in animals: categories and mechanisms. Biol Rev 69(2):207–231CrossRefPubMedGoogle Scholar
  24. Heyes CM (2009) Evolution, development and intentional control of imitation. Philos Trans R Soc B 364(1528):2293–2298CrossRefGoogle Scholar
  25. Heyes CM, Galef BG (1996) Social learning in animals: the roots of culture. Elsevier, AmsterdamGoogle Scholar
  26. Hoppitt W, Laland KN (2008) Social processes influencing learning in animals: a review of the evidence. Adv Stud Behav 38:105–165CrossRefGoogle Scholar
  27. Hoppitt W, Laland K (2013) Social learning: an introduction to mechanisms, methods, and models. Princeton University Press, PrincetonCrossRefGoogle Scholar
  28. Hoppitt W, Blackburn L, Laland KN (2007) Response facilitation in the domestic fowl. Anim Behav 73(2):229–238CrossRefGoogle Scholar
  29. Laland KN, Galef BG (2009) The question of animal culture. Harvard University Press, CambridgeGoogle Scholar
  30. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533CrossRefPubMedGoogle Scholar
  31. Ottoni EB, Izar P (2008) Capuchin monkey tool use: overview and implications. Evol Anthropl Issues, News Rev 17(4):171–178CrossRefGoogle Scholar
  32. Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25(26):6235–6242CrossRefPubMedGoogle Scholar
  33. Racey D, Young ME, Garlick D, Pham JNM, Blaisdell AP (2011) Pigeon and human performance in a multi-armed bandit task in response to changes in variable interval schedules. Learn Beahv 39(3):245–258CrossRefGoogle Scholar
  34. Reid AK, Chadwick CZ, Dunham M, Miller A (2001) The development of functional response units: the role of demarcating stimuli. J Expt Anal Behav 76(3):303–320CrossRefGoogle Scholar
  35. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning: current research and theory. Appleton-Century-Crofts, New York, pp 64–99Google Scholar
  36. Rogers AR (1988) Does Biology Constrain Culture? Am Anthropol 90(4):819–831CrossRefGoogle Scholar
  37. Sanz C, Call J, Morgan D (2009) Design complexity in termite-fishing tools of chimpanzees (Pan troglodytes). Biol Lett 5(3):293–296CrossRefPubMedCentralPubMedGoogle Scholar
  38. Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004) Temporal difference models describe higher-order learning in humans. Nature 429(6992):664–667CrossRefPubMedGoogle Scholar
  39. Stoinski TS, Whiten A (2003) Social learning by orangutans (Pongo abelii and Pongo pygmaeus) in a simulated food-processing task. J Comp Psychol 117(3):272CrossRefPubMedGoogle Scholar
  40. Sutton RS, Barto AG (1990) Time-derivative models of Pavlovian reinforcement. In: Gabriel M, Moore J (eds) Learning and computational neuroscience: foundations of adaptive networks. MIT Press, Cambridge, pp 497-537Google Scholar
  41. Sutton RS, Barto AG (1998) Reinforcement learning: AN introduction, vol 1. Cambridge Univ Press, CambridgeGoogle Scholar
  42. Tan M (1993) Multi-agent reinforcement learning: independent vs. cooperative agents. In: Proceedings of the tenth international conference on machine learning, pp 330–337Google Scholar
  43. Tennie C, Hedwig D, Call J, Tomasello M (2008) An experimental study of nettle feeding in captive gorillas. Am J Primatol 70(6):584–593CrossRefPubMedGoogle Scholar
  44. Terkel J (1996) Cultural transmission of feeding behavior. In: Heyes CM, Galef BG (eds) Social learning in animals: the roots of culture. Elsevier, Amsterdam, p 17CrossRefGoogle Scholar
  45. Terrace HS (2005) The simultaneous chain: a new approach to serial learning. Trends Cogn Sci 9(4):202–210CrossRefPubMedGoogle Scholar
  46. Thorndike EL (1898) Animal intelligence: an experimental study of the associative processes in animals. Psychol Monogr Gen Appl 2(4):1–109CrossRefGoogle Scholar
  47. Whiten A (1998) Imitation of the sequential structure of actions by chimpanzees (Pan troglodytes). J Comp Psychol 112(3):270CrossRefPubMedGoogle Scholar
  48. Whiten A, Goodall J, McGrew WC, Nishida T, Reynolds V, Sugiyama Y, Tutin CE, Wrangham RW, Boesch C (1999) Cultures in chimpanzees. Nature 399(6737):682–685CrossRefPubMedGoogle Scholar
  49. Zentall TR, Galef BG Jr (1988) Social learning: psychological and biological perspectives. Lawrence Erlbaum Associates, HillsdaleGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of BiologyUniversity of St AndrewsSt Andrews, FifeUK

Personalised recommendations