Using the Interaction Rhythm as a Natural Reinforcement Signal for Social Robots: A Matter of Belief

  • Antoine Hiolle
  • Lola Cañamero
  • Pierre Andry
  • Arnaud Blanchard
  • Philippe Gaussier
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6414)


In this paper, we present the results of a pilot study of a human robot interaction experiment where the rhythm of the interaction is used as a reinforcement signal to learn sensorimotor associations. The algorithm uses breaks and variations in the rhythm at which the human is producing actions. The concept is based on the hypothesis that a constant rhythm is an intrinsic property of a positive interaction whereas a break reflects a negative event. Subjects from various backgrounds interacted with a NAO robot where they had to teach the robot to mirror their actions by learning the correct sensorimotor associations. The results show that in order for the rhythm to be a useful reinforcement signal, the subjects have to be convinced that the robot is an agent with which they can act naturally, using their voice and facial expressions as cues to help it understand the correct behaviour to learn. When the subjects do behave naturally, the rhythm and its variations truly reflects how well the interaction is going and helps the robot learn efficiently. These results mean that non-expert users can interact naturally and fruitfully with an autonomous robot if the interaction is believed to be natural, without any technical knowledge of the cognitive capacities of the robot.


Facial Expression Autonomous Robot Reinforcement Signal Social Robot Human Partner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berthouze, L., Lungarella, M.: Motor skill acquisition under environmental perturbations: On the necessity of alternate freezing and freeing of degrees of freedom. Adaptive Behavior 12(1), 47–63 (2004)CrossRefGoogle Scholar
  2. 2.
    Oudeyer, P.-Y., Kaplan, F., Hafner, V.: Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 2(11) (2007)Google Scholar
  3. 3.
    Bowlby, J.: Attachment and loss. Attachment, vol. 1. Basics Books, New York (1969)Google Scholar
  4. 4.
    De Wolf, M.S., van IJzendoorn, M.H.: Sensitivity and attachment: A meta-analysis on parental antecedents of infant attachment. Child Development 68(4), 571–591 (1997)CrossRefGoogle Scholar
  5. 5.
    Tronick, E.: The neurobehavioral and social-emotional development of infants and children. WW Norton and Company, New York (2007)Google Scholar
  6. 6.
    Nadel, J., Soussignan, R., Canet, P., Libert, G., Grardin, P.: Two-month-old infants of depressed mothers show mild, delayed and persistent change in emotional state after non-contingent interaction. Infant Behavior and Development 28, 418–425 (2005)CrossRefGoogle Scholar
  7. 7.
    Nadel, J., Prepin, K., Okanda, M.: Experiencing contigency and agency: first step toward self-understanding. Interaction Studies 2, 447–462 (2005)Google Scholar
  8. 8.
    Hiolle, A., Cañamero, L.: Why should you care? an arousal-based model of exploratory behavior for autonomous robots. In: Bullock, S., Noble, J., Watson, R., Bedau, M.A. (eds.) Artificial Life XI: Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, pp. 242–248. MIT Press, Cambridge (2008)Google Scholar
  9. 9.
    Hiolle, A., Cañamero, L.: Developing sensorimotor associations through attachment bonds. In: Prince, C., Balkenius, C., Berthouze, L., Kozima, H., Littman, M. (eds.) Proc. 7th Intl. Wksp. on Epigenetic Robotics, pp. 45–52. Lund University Cognitive Studies (2007)Google Scholar
  10. 10.
    Hiolle, A., Bard, A.,, K., Cañamero, L.: Assessing human responses to different robot attachment profiles. In: Proceedings of the 18th International Symposium on Robot and Human Interactive Communication, pp. 251–257 (2009)Google Scholar
  11. 11.
    Andry, P., Gaussier, P., Moga, S., Banquet, J.-P., Nadel, J.: Learning and communication in imitation: An autonomous robot perspective. IEEE Transactions on Man, Systems and Cybernetics, Part A: Systems and humans 31(5), 431–442 (2001)CrossRefGoogle Scholar
  12. 12.
    Andry, P., Garnault, N., Gaussier, P.: Using the interaction rhythm to build an internal reinforcement signal: a tool for intuitive hri. In: Prince, C., Balkenius, C., Berthouze, L., Kozima, H., Littman, M. (eds.) Proceedings of the Ninth Int. Conf. on Epigenetic Robotics, Lund University Cognitive Studies (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Antoine Hiolle
    • 1
  • Lola Cañamero
    • 1
  • Pierre Andry
    • 2
  • Arnaud Blanchard
    • 2
  • Philippe Gaussier
    • 2
  1. 1.Adaptive Systems Research Group, School of Computer ScienceUniversity of HertfordshireEngland
  2. 2.ETIS, ENSEAUniversite de Cergy-Pontoise, CNRSFrance

Personalised recommendations