Skip to main content
SpringerLink
Log in

Emotional Understanding and Behavior Learning for Haru via Social Reinforcement Learning

  • Conference paper
  • First Online:
Social Robotics (ICSR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14454))

Included in the following conference series:

  • 324 Accesses

Abstract

As a new type of human companion, social robots are becoming more and more popular and expected to being fully integrated with human daily life in the near future. Being able to correctly perceive the emotions of users and react to it can increase the sense of trust, affinity, and social presence of human-robot interaction. In this paper, we propose a human-centered reinforcement learning strategy to train social robots to achieve autonomous emotion understanding and behavior shaping. Our whole study was conducted on the social robot Haru, which has a large library of routines to express different emotions. Our experimental results show that autonomous emotion understanding and behavior shaping of social robots can be achieved through continuous interaction with humans.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Appel, M., Izydorczyk, D., Weber, S., Mara, M., Lischetzke, T.: The uncanny of mind in a machine: humanoid robots as tools, agents, and experiencers. Comput. Hum. Behav. 102, 274–286 (2020)

    Article  Google Scholar 

  2. Cramer, H., Goddijn, J., Wielinga, B., Evers, V.: Effects of (in) accurate empathy and situational valence on attitudes towards robots. In: Proceedings of 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 141–142. IEEE (2010)

    Google Scholar 

  3. Leite, I., Castellano, G., Pereira, A., Martinho, C., Paiva, A.: Empathic robots for long-term interaction: evaluating social presence, engagement and perceived support in children. Int. J. Soc. Robot. 6, 329–341 (2014)

    Article  Google Scholar 

  4. Merla, A.: Thermal expression of intersubjectivity offers new possibilities to human-machine and technologically mediated interactions. Front. Psychol. 5, 802 (2014)

    Article  Google Scholar 

  5. Filippini, C., et al.: Facilitating the child–robot interaction by endowing the robot with the capability of understanding the child engagement: the case of Mio Amico robot. Int. J. Soc. Robot. 13(4), 677–689 (2020). https://doi.org/10.1007/s12369-020-00661-w

    Article  Google Scholar 

  6. Melinte, D.O., Vladareanu, L.: Facial expressions recognition for human-robot interaction using deep convolutional neural networks with rectified Adam optimizer. Sensors 20(8), 2393 (2020)

    Article  Google Scholar 

  7. Eyam, A.T., Mohammed, W.M., Lastra, J.L.M.: Emotion-driven analysis and control of human-robot interactions in collaborative applications. Sensors 21(14), 4626 (2021)

    Google Scholar 

  8. Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., Movellan, J.R.: The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014)

    Article  Google Scholar 

  9. Cockburn, J., Bartlett, M., Tanaka, J., Movellan, J., Pierce, M., Schultz, R.: SmileMaze: a tutoring system in real-time facial expression perception and production in children with autism spectrum disorder. In: ECAG 2008 Workshop Facial and Bodily Expressions for Control and Adaptation of Games, vol. 3. Amsterdam (2008)

    Google Scholar 

  10. Liu, Z., et al.: A facial expression emotion recognition based human-robot interaction system. IEEE CAA J. Autom. Sinica 4(4), 668–676 (2017)

    Article  Google Scholar 

  11. Fukuda, T., Tachibana, D., Arai, F., Taguri, J., Nakashima, M., Hasegawa, Y.: Human-robot mutual communication system. In: Proceedings of 10th IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN), pp. 14–19. IEEE (2001)

    Google Scholar 

  12. Leo, M., et al.: Automatic emotion recognition in robot-children interaction for ASD treatment. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 145–153 (2015)

    Google Scholar 

  13. Göngör, F., Tutsoy, Ö., Barkana, D.E., Köse, H.: An emotion analysis algorithm and implementation to NAO humanoid robot. In: Proceedings of International Conference on Technology, Engineering and Science (IConTES), vol. 7. https://www.researchgate.net/publication/321704164 (2021)

  14. May, A.D., Lotfi, A., Langensiepen, C., Lee, K., Acampora, G.: Human emotional understanding for empathetic companion robots. In: Angelov, P., Gegov, A., Jayne, C., Shen, Q. (eds.) Advances in Computational Intelligence Systems. AISC, vol. 513, pp. 277–285. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-46562-3_18

    Chapter  Google Scholar 

  15. Heredia, J., Cardinale, Y., Dongo, I., Aguilera, A., Diaz-Amado, J.: Multimodal emotional understanding in robotics. In: Workshops at 18th International Conference on Intelligent Environments (IE2022). IOS Press (2022)

    Google Scholar 

  16. Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787–796 (2015)

    Google Scholar 

  17. Polyak, A., Wolf, L.: Channel-level acceleration of deep face representations. IEEE Access 3, 2163–2175 (2015)

    Article  Google Scholar 

  18. Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. 13(3), 1195–1215 (2020)

    Article  MathSciNet  Google Scholar 

  19. Bagheri, E., Esteban, P.G., Cao, H.-L., Beir, A.D., Lefeber, D., Vanderborght, B.: An autonomous cognitive empathy model responsive to users’ facial emotion expressions. ACM Trans. Interact. Intell. Syst. (TIIS) 10(3), 1–23 (2020)

    Article  Google Scholar 

  20. Chen, L., Li, M., Su, W., Wu, M., Hirota, K., Pedrycz, W.: Adaptive feature selection-based Adaboost-KNN with direct optimization for dynamic emotion recognition in human-robot interaction. IEEE Trans. Emerg. Top. Comput. Intell. 5(2), 205–213 (2019)

    Article  Google Scholar 

  21. Chen, L., Zhou, M., Su, W., Wu, M., She, J., Hirota, K.: Softmax regression based deep sparse autoencoder network for facial emotion recognition in human-robot interaction. Inf. Sci. 428, 49–61 (2018)

    Article  MathSciNet  Google Scholar 

  22. Mnih, V., et al.: Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

  23. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)

    Article  MATH  Google Scholar 

  24. Sandry, E., Gomez, R., Nakamura, K.: Art, design and communication theory in creating the communicative social robot ‘haru’. Front. Robot. AI 8, 577107 (2021)

    Article  Google Scholar 

  25. Brock, H., Sabanovic, S., Nakamura, K., Gomez, R.: Robust real-time hand gestural recognition for non-verbal communication with tabletop robot Haru. In: Proceedings of 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 891–898. IEEE (2020)

    Google Scholar 

  26. Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ocean University of China, Qingdao, China

    Lei Zhang, Chuanxiong Zheng, Hui Wang & Guangliang Li

  2. Honda Research Institute Japan Co. Ltd., Wako, Japan

    Eric Nichols & Randy Gomez

Authors
  1. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chuanxiong Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric Nichols
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Randy Gomez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangliang Li .

Editor information

Editors and Affiliations

  1. Qatar University, Doha, Qatar

    Abdulaziz Al Ali

  2. Qatar University, Doha, Qatar

    John-John Cabibihan

  3. Qatar University, Doha, Qatar

    Nader Meskin

  4. University of Naples Federico II, Napoli, Italy

    Silvia Rossi

  5. Qingdao University, Qingdao, China

    Wanyue Jiang

  6. The University of Alabama, Tuscaloosa, AL, USA

    Hongsheng He

  7. National University of Singapore, Queenstown, Singapore

    Shuzhi Sam Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, L., Zheng, C., Wang, H., Nichols, E., Gomez, R., Li, G. (2024). Emotional Understanding and Behavior Learning for Haru via Social Reinforcement Learning. In: Ali, A.A., et al. Social Robotics. ICSR 2023. Lecture Notes in Computer Science(), vol 14454. Springer, Singapore. https://doi.org/10.1007/978-981-99-8718-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8718-4_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8717-7

  • Online ISBN: 978-981-99-8718-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation