Skip to main content
SpringerLink
Log in

Motivation in Natural and Artificial Agents

  • Chapter
  • First Online:
Motivated Reinforcement Learning

  • 1214 Accesses

Abstract

Motivation is “the reason one has for acting or behaving in a particular way”. Motivation in humans has been studied from many different perspectives: biological, psychological, social, and so on. In contrast, motivation in artificial systems is a relatively new idea. In order to use motivation as a basis for designing adaptive characters that can evolve in complex, dynamic virtual environments, we consider models of motivation that are inspired by concepts from both natural systems and artificial intelligence. Specifically, theories from the psychological study of motivation are used as the basis for designing agents that can adapt by exhibiting problem-finding behaviour; that is, identifying new tasks on which to focus their attention. Computational models of motivation are combined with models of reinforcement learning [1] from the field of machine learning to develop agents that can adapt to new problems by learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R.S. Sutton and A.G. Barto, Reinforcement learning: an introduction, The MIT Press Cambridge, Massachusetts, London, England, 2000.

    Google Scholar 

  2. D.G. Mook, Motivation: the organization of action, W. W. Norton and Company, New York, 1987.

    Google Scholar 

  3. R.G. Geen, W.W. Beatty and R.M. Arkin, Human motivation: physiological, behavioral and social approaches, Allyn and Bacon, Massachusetts, 1984.

    Google Scholar 

  4. E.R. Kandel, J.H. Schwarz and T.M. Jessell, Essentials of neural science and behaviour, Appleton and Lang, Norwalk, 1995.

    Google Scholar 

  5. C. Alderfer, Existence, relatedness and growth, Free Press, New York, 1972.

    Google Scholar 

  6. A. Maslow, Motivation and personality, Harper, New York, 1954.

    Google Scholar 

  7. C.L. Hull, Principles of behavior, Appleton-Century-Crofts, New York, 1943.

    Google Scholar 

  8. C.L. Hull, A behavior system: an introduction to behavior theory concerning the individual organism, Yale University Press, New Haven, 1952.

    Google Scholar 

  9. O. Avila-Garcia and L. Canamero, Comparison of behaviour selection architectures using viability indicators, The EPSRC/BBSRC International Workshop on Biologically Inspired Robotics: The Legacy of W. Grey Walter, HP Labs, Bristol, UK, pp. 8–93, 2002.

    Google Scholar 

  10. L. Canamero, Modelling motivations and emotions as a basis for intelligent behaviour. In: W.L. Johnson (Eds.), The First International Symposium on Autonomous Agents, ACM Press, New York, NY, pp. 148–155, 1997.

    Chapter  Google Scholar 

  11. C. Gershenson, Artificial Societies of Intelligent Agents, Engineering, Fundacion Arturo Rosenblueth, Mexico, 2001.

    Google Scholar 

  12. R. Aylett, A. Coddington and G. Petley, Agent-based continuous planning, The 19th Workshop of the UK Planning and Scheduling Special Interest Group (PLANSIG 2000), 2000.

    Google Scholar 

  13. M. Luck and M. d'Inverno, Motivated behaviour for goal adoption. In: C. Zhang and D. Lukose (Eds.), Multi-Agent Systems: Theories, Languages and Applications, Proceedings of the Fourth Australian Workshop on Distributed Artificial Intelligence, Springer-Verlag, pp. 58–73, 1998.

    Google Scholar 

  14. T.J. Norman and D. Long, Goal creation in motivated agents, Intelligent agents: theories, architectures and languages, Springer-Verlag, pp. 277–290, 1995.

    Google Scholar 

  15. M. Schmill and P. Cohen, A motivational system that drives the development of activity, AAMAS, ACM, Bologna, Italy, pp. 358–359, 2002.

    Google Scholar 

  16. D. McFarland, Animal behaviour, Longman, England, 1995.

    Google Scholar 

  17. D. Bindra, A unified account of classical conditioning and operant training. In: A. Black and W. Prokasy (Eds.), Classical conditioning – current theory and research, Appleton-Century-Crofts, New York, USA, pp. 453–481, 1974.

    Google Scholar 

  18. W. Wundt, Principles of physiological psychology, Macmillan, New York, 1910.

    Google Scholar 

  19. D.E. Berlyne, Novelty, complexity and hedonic value. Perception and Psychophysics 8:279–286, 1970.

    Google Scholar 

  20. D.E. Berlyne, Aesthetics and psychobiology, Prentice-Hall, Englewood Cliffs, New Jersey, 1971.

    Google Scholar 

  21. D.E. Berlyne, Exploration and curiosity. Science 153:25–33, 1966.

    Article  Google Scholar 

  22. S. Marsland, U. Nehmzow and J. Shapiro, A real-time novelty detector for a mobile robot, EUREL European Advanced Robotics Systems Masterclass and Conference, 2000.

    Google Scholar 

  23. J.A. Easterbrook, The effect of emotion on cue utilization and the organisation of behavior. Psychological Review 66:183–201, 1959.

    Article  Google Scholar 

  24. D. Sperber and D. Wilson, Relevance: communication and cognition, Blackwell Publishing, Oxford, 1995.

    Google Scholar 

  25. L. Fonar and P. Maes, Paying attention to what’s important: using focus of attention to improve unsupervised learning, The Third International Conference on the Simulation of Adaptive Behavior (SAB94), Brighton, England, 1994.

    Google Scholar 

  26. J.C. Stanley, Computer simulation of a model of habituation. Nature 26:146–148, 1976.

    Article  Google Scholar 

  27. T. Kohonen, Self-organization and associative memory, Springer, Berlin, 1993.

    Google Scholar 

  28. R. Saunders and J.S. Gero, Curious agents and situated design evaluations. In: J.S. Gero and F.M.T. Brazier (Eds.), Agents in Design, Key Centre of Design Computing and Cognition, University of Sydney, pp. 133–149, 2002.

    Google Scholar 

  29. R. Saunders and J.S. Gero, Situated design simulations using curious agents. AIEDAM 18(2):153–161, 2004.

    Article  Google Scholar 

  30. D. Helbing and P. Molnar, Social force model for pedestrian dynamics. Physical Review E 51:4282–4286, 1995.

    Article  Google Scholar 

  31. R. Saunders and J.S. Gero, Designing for interest and novelty: motivating design agents, CAAD Futures 2001, Kluwer, Dordrecht, pp. 725–738, 2001.

    Google Scholar 

  32. J. Schmidhuber, A possibility for implementing curiosity and boredom in model-building neural controllers. In: J.A. Meyer and S.W. Wilson (Eds.), The International Conference on Simulation of Adaptive Behavior: From Animals to Animats, MIT Press/Bradford Books, pp. 222–227, 1991.

    Google Scholar 

  33. D. Lenat, AM: An artificial intelligence approach to discovery in mathematics, Computer Science, Stanford University, 1976.

    Google Scholar 

  34. J. Schmidhuber, What's interesting, Technical Report, Lugano, Switzerland, 1997.

    Google Scholar 

  35. P.-Y. Oudeyer and F. Kaplan, Intelligent adaptive curiosity: a source of self-development, Fourth International Workshop on Epigenetic Robotics, Lund University, pp. 127–130, 2004.

    Google Scholar 

  36. P.-Y. Oudeyer, F. Kaplan and V. Hafner, Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 11(2):265–286, 2007.

    Article  Google Scholar 

  37. F. Kaplan and P.-Y. Oudeyer, Motivational principles for visual know-how development. In: C.G. Prince, L. Berthouze, H. Kozima, D. Bullock, G. Stojanov and C. Balkenius (Eds.), Proceedings of the 3rd international workshop on Epigenetic Robotics: Modelling cognitive development in robotic systems, Lund University Cognitive Studies, pp. 73–80, 2003.

    Google Scholar 

  38. E.C. Tolman, Purposive behavior in animals and men, Century, New York, 1932.

    Google Scholar 

  39. B.F. Skinner, Behavior of organisms, Appleton-Century-Crofts, New York, 1938.

    Google Scholar 

  40. J.W. Atkinson and N.T. Feather, A theory of achievement motivation, John Wiley, New York, 1966.

    Google Scholar 

  41. J.W. Atkinson and J.O. Raynor, Motivation and achievement, V. H. Winston, Washington D. C, 1974.

    Google Scholar 

  42. J.O. Raynor, Future orientation and motivation of immediate activity: an elaboration of the theory of achievement motivation. Psychological Review 76:606–610, 1969.

    Article  Google Scholar 

  43. F. Heider, The psychology of interpersonal relations, Wiley, New York, 1958.

    Book  Google Scholar 

  44. R.W. White, Motivation reconsidered: The concept of competence. Psychological Review 66:297–333, 1959.

    Article  Google Scholar 

  45. E. Deci and R. Ryan, Intrinsic motivation and self-determination in human behavior, Plenum Press, New York, 1985.

    Google Scholar 

  46. J.M. Hunt, Implications of sequential order and hierarchy in early psychological development. Exceptional Infant 3, 1975.

    Google Scholar 

  47. S. Singh, A.G. Barto and N. Chentanez, Intrinsically motivated reinforcement learning, Advances in Neural Information Processing Systems (NIPS) 17:1281–1288, 2005.

    Google Scholar 

  48. D. Precup, R. Sutton and S. Singh, Theoretical results on reinforcement learning with temporally abstract options, The 10th European Conference on Machine Learning, Chemnitz, Germany, Springer Verlag, pp. 382–393, 1998.

    Google Scholar 

  49. M. Csikszentmihalyi, Creativity: Flow and the psychology of discovery and invention, HarperCollins Publisher, New York, 1996.

    Google Scholar 

  50. R. Saunders and J.S. Gero, The digital clockwork muse: a computational model of aesthetic evolution, The AISB'01 Symposium on Artificial Intelligence and Creativity in Arts and Science, SSAISB, 2001.

    Google Scholar 

  51. C. Darwin, The origin of the species, John Murray, London, 1859.

    Google Scholar 

  52. R.V. Gusarev, M.S. Burtsev and V.G. Red'ko, Alife model of evolutionary emergence of purposeful adaptive behavior, European Conference on Artificial Life, 2001.

    Google Scholar 

  53. R. Stagner, Homeostasis, discrepancy, dissonance: a theory of motives and motivation. Motivation and Emotion 1:103–138, 1977.

    Article  Google Scholar 

  54. R.A. Brooks, How to build complete creatures rather than isolated cognitive simulators. In: K. VanLehn (Ed.), Architectures for Intelligence, Lawrence Erlbaum Associates, New Jersey, pp. 225–239, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Technology and Electrical Engineering, University of New South Wales, Australian Defence Force Academy, Building 16, Northcott Drive, Canberra, ACT, 2600, Australia

    Dr. Kathryn E. Merrick

  2. Faculty of Architecture, Design, and Planning, University of Sydney, 148 City Road, Sydney, NSW, 2006, Australia

    Prof. Mary Lou Maher

Authors
  1. Dr. Kathryn E. Merrick
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prof. Mary Lou Maher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathryn E. Merrick .

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Merrick, K.E., Maher, M.L. (2009). Motivation in Natural and Artificial Agents. In: Motivated Reinforcement Learning. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89187-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-89187-1_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-89186-4

  • Online ISBN: 978-3-540-89187-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation