Design of a Control Architecture for Habit Learning in Robots

  • Erwan Renaudo
  • Benoît Girard
  • Raja Chatila
  • Mehdi Khamassi
Conference paper

DOI: 10.1007/978-3-319-09435-9_22

Part of the Lecture Notes in Computer Science book series (LNCS, volume 8608)
Cite this paper as:
Renaudo E., Girard B., Chatila R., Khamassi M. (2014) Design of a Control Architecture for Habit Learning in Robots. In: Duff A., Lepora N.F., Mura A., Prescott T.J., Verschure P.F.M.J. (eds) Biomimetic and Biohybrid Systems. Living Machines 2014. Lecture Notes in Computer Science, vol 8608. Springer, Cham

Abstract

Researches in psychology and neuroscience have identified multiple decision systems in mammals, enabling control of behavior to shift with training and familiarity of the environment from a goal-directed system to a habitual system. The former relies on the explicit estimation of future consequences of actions through planning towards a particular goal, which makes decision time longer but produces rapid adaptation to changes in the environment. The latter learns to associate values to particular stimulus-response associations, leading to quick reactive decision- making but slow relearning in response to environmental changes. Computational neuroscience models have formalized this as a coordination of model-based and model-free reinforcement learning. From this inspiration we hypothesize that it could enable robots to learn habits, detect when these habits are appropriate and thus avoid long and costly computations of the planning system. We illustrate this in a simple repetitive cube-pushing task on a conveyor belt, where a speed-accuracy trade-off is required. We show that the two systems have complementary advantages in these tasks, which can be combined for performance improvement.

Keywords

Adaptive Behaviour Habit Learning Reinforcement Learning Robotic Architecture 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Erwan Renaudo
    • 1
    • 2
  • Benoît Girard
    • 1
    • 2
  • Raja Chatila
    • 1
    • 2
  • Mehdi Khamassi
    • 1
    • 2
  1. 1.Institut des Systèmes Intelligents et de RobotiqueSorbonne Universités, UPMC Univ Paris 06, UMR 7222ParisFrance
  2. 2.Institut des Systèmes Intelligents et de RobotiqueCNRS, UMR 7222ParisFrance

Personalised recommendations