Neural Computing and Applications

, Volume 24, Issue 1, pp 117–124 | Cite as

Robots learn to dance through interaction with humans

  • Qinggang MengEmail author
  • Ibrahim Tholley
  • Paul W. H. Chung


In this paper, we investigated an approach for robots to learn to adapt dance actions to human’s preferences through interaction and feedback. Human’s preferences were extracted by analysing the common action patterns with positive or negative feedback from the human during robot dancing. By using a buffering technique to store the dance actions before a feedback, each individual’s preferences can be extracted even when a reward is received late. The extracted preferred dance actions from different people were then combined to generate improved dance sequences, i.e. performing more of what was preferred and less of that was not preferred. Together with Softmax action-selection method, the Sarsa reinforcement learning algorithm was used as the underlining learning algorithm and to effectively control the trade-off between exploitation of the learnt dance skills and exploration of new dance actions. The results showed that the robot learnt, using interactive reinforcement learning, the preferences of human partners, and the dance improved with the extracted preferences from more human partners.


Robot dancing Robot learning Robot adaptation Robot interaction with humans 


  1. 1.
    Aucouturier JJ (2008) Cheek to chip: dancing robots and AI’s future. IEEE Intell Syst 23(2):74–84CrossRefGoogle Scholar
  2. 2.
    Austermann A, Yamada S (2008) “Good robot”, “bad robot”-analyzing users’ feedback in a human-robot teaching task. In: The 17th IEEE international symposium on robot and human interactive communication, pp 41–46Google Scholar
  3. 3.
    Cyberbotics (2011) Webots 6 for fast prototyping and simulation of mobile robots [online]. Available from:
  4. 4.
    Dozier G (2001) Evolving robot behavior via interactive evolutionary computation: from real-world to simulation. In: Proceedings of the 2001 ACM symposium on applied computing, pp 340–344Google Scholar
  5. 5.
    Jens H, Peer A, Buss M (2010) Synthesis of an interactive haptic dancing partner. In: The 19th IEEE international symposium on robot and human interactive communication, Viareggio, Italy, 12–15 Sept 2010, pp 527–532Google Scholar
  6. 6.
    Kober J, Bagnell JAD, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Rob Res (in press)Google Scholar
  7. 7.
    Leopold T, Kern-Isberner G, Peters G (2008) Belief revision with reinforcement learning for interactive object recognition. In: Proceedings of 18th European conference on artificial intelligence, pp 65–69Google Scholar
  8. 8.
    Liu F, Su J (2004) Visual learning framework based on reinforcement learning. In: Fifth World Congress on Intelligent Control and Automation, 6:4865–4868Google Scholar
  9. 9.
    Peralta R, Kaochar T, Fasel I, Morrison C, Walsh T, Cohen P (2011) Challenges to decoding the intention behind natural instruction. In: Proceedings of RO-MAN, 2011, pp 113–118Google Scholar
  10. 10.
    Santiago C, Oliveira J, Reis L, Sousa A (2011) Autonomous robot dancing synchronized to musical rhythmic stimuli. In: 2011 6th Iberian conference on information systems and technologies (CISTI), pp 1–6Google Scholar
  11. 11.
    Shiratori T, Ikeuchi K (2008) Synthesis of dance performance based on analyses of human motion and music. IPSJ Trans Comput Vis Image Media 1(1):34–47Google Scholar
  12. 12.
    Shiratori T, Nakazawa A, Ikeuchi K (2006) Dancing-to-music character animation. Comput Graph Forum Proc Eurograph 2006 25(3):449–458CrossRefGoogle Scholar
  13. 13.
    Solis J, Chida K, Suefuji K, Takanishi A (2005) Improvements of the sound perception processing of the anthropomorphic flutist robot (WF-4R) to effectively interact with humans. In: Proceedings of IEEE international workshop on robots and human interactive communication, Nashville, USA, pp 450–455Google Scholar
  14. 14.
    Suay HB, Chernova S (2011) Effect of human guidance and state space size on interactive reinforcement learning. In: 20th IEEE international symposium on robot and human interactive communication, 31 July 2011–3 Aug 2011, pp 1–6Google Scholar
  15. 15.
    Suga Y, Ikuma Y, Nagao D, Ogata T, Sugano S (2005) Interactive evolution of human-robot communication in real world. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp 1438–1443Google Scholar
  16. 16.
    Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT press, Bradford Books, CambridgeGoogle Scholar
  17. 17.
    Tanaka F, Movellan JR, Fortenberry B, Aisaka K (2006) Daily HRI evaluation at a classroom environment: reports from dance interaction experiments. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction, pp 3–9Google Scholar
  18. 18.
    Tholley I (2012) Towards a framework to make robots learn to dance. PhD thesis, Loughborough University, UKGoogle Scholar
  19. 19.
    Thomaz A, Breazeal C (2007) Asymmetric interpretations of positive and negative human feedback for a social learning agent. In: The 16th IEEE international symposium on robot and human interactive communication, pp 720–725Google Scholar
  20. 20.
    Thomaz AL, Hoffman G, Breazeal C (2005) Real-time interactive reinforcement learning for robots. In: AAAI 2005 workshop on human comprehensible machine learningGoogle Scholar
  21. 21.
    Vircikova M, Sincak P (2010) Dance choreography design of humanoid robots using interactive evolutionary computation. In: 3rd Workshop for Young Researchers: Human Friendly Robotics for Young ResearchersGoogle Scholar
  22. 22.
    Wang H, Kosuge K (2012) Attractor design and prediction-based adaption for a robot waltz dancer in physical human-robot interaction. In: Proceedings of the 2012 World Congress on intelligent control and automation, pp 3810–3815Google Scholar
  23. 23.
    Wang H, Kosuge K (2012) Understanding and reproducing waltz dancers’ body dynamics in physical human-robot interaction. In: Proceedings of 2012 IEEE international conference robotics and automation (ICRA), pp 3134–3140Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Qinggang Meng
    • 1
    Email author
  • Ibrahim Tholley
    • 1
  • Paul W. H. Chung
    • 1
  1. 1.Department of Computer ScienceLoughborough UniversityLoughboroughUK

Personalised recommendations