Adaptive Motivation System Under Modular Reinforcement Learning for Agent Decision-Making Modeling of Biological Regulation

  • Amine Chohra
  • Kurosh Madani
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10449)


In this paper, an adaptive motivation system under modular reinforcement learning is suggested for agent decision-making modeling of biological regulation. For this purpose, first, main concepts of drives, rewards, action selection under modular reinforcement learning as well as an adaptive priority process are developed. Second, experiments and results are presented and analyzed demonstrating the efficiency of the suggested concepts. Finally, a discussion is given in conclusion with regard to related works. The obtained results demonstrate how the suggested adaptive motivation system can be used by an agent learning (on-line) to select appropriate actions, during a navigation task from a starting position to a goal position (external goal), i.e., in each moving step, in order to reach an external goal as well as to satisfy internal goals (drives such as hunger, thirst, …); predicting a promising result in future to demonstrate how the nature of the interaction (stimulation-drive, social-drive, …) influences the agent behavior.


Decision-making Adaptive goal-directed behavior Agent–environment interactions Motivation Modular reinforcement learning Action selection 


  1. 1.
    Hull, C.L.: Principles of Behavior, An Introduction to Behavior Theory. D. Appleton-Century Co, New York (1943)Google Scholar
  2. 2.
    Singh, S., Lewis, R.L., Barto, A.G., Sorg, J.: Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Trans. Auton. Ment. Dev. 2(2), 70–82 (2010)CrossRefGoogle Scholar
  3. 3.
    Halliday, T.: Motivation. In: Halliday, T.R., Slater, P.J.B. (eds.) Causes and Effects. Blackwell Scientific, Oxford (1983)Google Scholar
  4. 4.
    Cannon, W.B.: The Wisdom of the Body. W. W. Norton, New York (1932)Google Scholar
  5. 5.
    Savage, T.: Artificial motives: a review of motivation in artificial creatures. Connect. Sci. 12, 211–277 (2000)CrossRefGoogle Scholar
  6. 6.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  7. 7.
    Brooks, R.A.: The role of learning in autonomous robots. In: Proceedings of the Fourth Annual Workshop on Computational Learning Theory (COLT 1991), pp. 05–10 (1991)CrossRefGoogle Scholar
  8. 8.
    Chohra, A., Madani, K.: Biological regulation and psychological mechanisms models of adaptive decision-making behaviors: drives, emotions, and personality. In: Nguyen, N.-T., Manolopoulos, Y., Iliadis, L., Trawiński, B. (eds.) ICCCI 2016. LNCS, vol. 9875, pp. 412–422. Springer, Cham (2016). doi: 10.1007/978-3-319-45243-2_38CrossRefGoogle Scholar
  9. 9.
    Warren, W.H.: The dynamics of perception and action. Psychol. Rev. 113(2), 358–389 (2006)CrossRefGoogle Scholar
  10. 10.
    Konidaris, G., Barto, A.: An adaptive robot motivational system. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS, vol. 4095, pp. 346–356. Springer, Heidelberg (2006). doi: 10.1007/11840541_29CrossRefGoogle Scholar
  11. 11.
    Breazeal, C., Brooks, R.A.: Robot emotion: a functional perspective. In: Fellous, J.-M., Arbib, M. (eds.) MIT Press, pp. 271–310 (2005)CrossRefGoogle Scholar
  12. 12.
    Berridge, K.C.: Rewards learning: reinforcement, incentives, and expectations. Psychol. Learn. Motiv. 40, 223–278 (2001). Academic PressCrossRefGoogle Scholar
  13. 13.
    Sprague, N., Ballard, D.: Multiple-goal reinforcement learning with modular Sarsa(0). Technical report 798, The University of Rochester (2004)Google Scholar
  14. 14.
    Bhat, S., Isbell Jr., C.L., Mateas, M.: On the difficulty of modular reinforcement learning for real-world partial programming. In: American Association Artificial Intelligence, pp. 318–323 (2006)Google Scholar
  15. 15.
    Humphrys, M.: Action selection methods using reinforcement learning. Ph.D. dissertation, University of Cambridge, Technical report (1997)Google Scholar
  16. 16.
    Karlsson, J.: Learning to solve multiple goals. Ph.D. thesis, University of Rochester (1997)Google Scholar
  17. 17.
    Cos, I., Canamero, L., Hayes, G.M., Gillies, A.: Hedonic value: enhancing adaptation for motivated agents. Adapt. Behav. 21(6), 465–483 (2013)CrossRefGoogle Scholar
  18. 18.
    Ashby, W.R.: Design for a Brain. Chapman & Hall, London (1952)zbMATHGoogle Scholar
  19. 19.
    Meyer, J.-A.: Artificial life and the animat approach to artificial intelligence. Artif. Intell. (1995). Academic PressGoogle Scholar
  20. 20.
    Kompella, V.R., Kazerounian, S., Schmidhuber, J.: An anti-hebbian learning rule to represent drive motivations for reinforcement learning. In: del Pobil, A.P., Chinellato, E., Martinez-Martin, E., Hallam, J., Cervera, E., Morales, A. (eds.) SAB 2014. LNCS, vol. 8575, pp. 176–187. Springer, Cham (2014). doi: 10.1007/978-3-319-08864-8_17CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Images, Signals, and Intelligent Systems Laboratory (LISSI/EA 3956), Paris-East University (UPEC), Senart Institute of TechnologyLieusaintFrance

Personalised recommendations