Skip to main content

Emotional Intelligence in Robotics: A Scoping Review

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1410)

Abstract

Research suggests that emotionally responsive machines that can simulate empathy increase de acceptance of users towards them, as the feeling of affinity towards the machine reduces negative perceptual feedback. In order to endow a robot with emotional intelligence, it must be equipped with sensors capable of capturing users’ emotions (sense), appraisal captured emotions to regulate its internal state (compute), and finally perform tasks where actions are regulated by the computed “emotional” state (act). However, despite the impressive progress made in recent years in terms of artificial intelligence, speech recognition and synthesis, computer vision and many other disciplines directly and indirectly related to artificial emotional recognition and behavior, we are still far from being able to endow robots with the empathic capabilities of a human being. This article aims to give an overview of the implications of introducing emotional intelligence in robotic constructions by discussing recent advances in emotional intelligence in robotics.

Keywords

  • Human-robot-interaction
  • Emotional intelligence
  • Review

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-87687-6_7
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-87687-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   199.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.

References

  1. Aranha, R.V., Corrêa, C.G., Nunes, F.L.S.: Adapting software with affective computing: a systematic review. IEEE Trans. Affect. Comput. 1 (2019). https://doi.org/10.1109/TAFFC.2019.2902379

  2. Yadegaridehkordi, E., Noor, N.F.B.M., Ayub, M.N.B., Affal, H.B., Hussin, N.B.: Affective computing in education: a systematic review and future research. Comput. Educ. 142, 103649 (2019). https://doi.org/10.1016/j.compedu.2019.103649

  3. Dudek, M., Baisch, S., Knopf, M., Kolling, T.: This isn’t me!: the role of age-related self- and user images for robot acceptance by elders. Int. J. Soc. Robot. (2020). https://doi.org/10.1007/s12369-020-00678-1

    CrossRef  Google Scholar 

  4. Mele, C., et al.: Understanding robot acceptance/rejection: the SAR model. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 470–475 (2020). https://doi.org/10.1109/RO-MAN47096.2020.9223577

  5. Meissner, A., Trübswetter, A., Conti-Kufner, A.S., Schmidtler, J.: Friend or foe? Understanding assembly workers 2019; acceptance of human-robot collaboration. ACM Trans. Hum.-Robot Interacti. 10(1), 3:1–3:30. https://doi.org/10.1145/3399433

  6. Savery, R., Weinberg, G.: A survey of robotics and emotion: classifications and models of emotional interaction. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 986–993 (2020). https://doi.org/10.1109/RO-MAN47096.2020.9223536

  7. Salovey, P., Mayer, J.D.: Emotional intelligence. Imagin. Cogn. Pers. 9(3), 185–211 (1990). https://doi.org/10.2190/DUGG-P24E-52WK-6CDG

    CrossRef  Google Scholar 

  8. What is a robot? - ROBOTS: your guide to the world of robotics. (n.d.). https://robots.ieee.org/learn/what-is-a-robot/. Accessed: 28 Apr 2021

  9. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980). https://doi.org/10.1037/h0077714

    CrossRef  Google Scholar 

  10. Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018). https://doi.org/10.3390/s18072074

    CrossRef  Google Scholar 

  11. van der Kruk, E., Reijne, M.M.: Accuracy of human motion capture systems for sport applications; state-of-the-art review. Eur. J. Sport Sci. 18(6), 806–819 (2018). https://doi.org/10.1080/17461391.2018.1463397

    CrossRef  Google Scholar 

  12. Rohlfing, M.L., Buckley, D.P., Piraquive, J., Stepp, C.E., Tracy, L.F.: Hey Siri: How effective are common voice recognition systems at recognizing dysphonic voices? The Laryngoscope (2021). https://doi.org/10.1002/lary.29082

  13. Samadiani, N., et al.: A review on automatic facial expression recognition systems assisted by multimodal sensor data. Sensors 19(8), 1863 (2019). https://doi.org/10.3390/s19081863

    CrossRef  Google Scholar 

  14. Khalil, R.A., Jones, E., Babar, M.I., Jan, T., Zafar, M.H., Alhussain, T.: Speech emotion recognition using deep learning techniques: a review. IEEE Access 7, 117327–117345 (2019). https://doi.org/10.1109/ACCESS.2019.2936124

    CrossRef  Google Scholar 

  15. Abdullah, S.M.S.A., Ameen, S.Y.A., Sadeeq, M.A.M., Zeebaree, S.: Multimodal emotion recognition using deep learning. J. Appl. Sci. Technol. Trends 2(2), 52–58 (2021). https://doi.org/10.38094/jastt20291

  16. Tzirakis, P., Chen, J., Zafeiriou, S., Schuller, B.: End-to-end multimodal affect recognition in real-world environments. Inf. Fusion 68, 46–53 (2021). https://doi.org/10.1016/j.inffus.2020.10.011

    CrossRef  Google Scholar 

  17. Li, S., et al.: Bi-modality fusion for emotion recognition in the wild. In: 2019 International Conference on Multimodal Interaction, pp. 589–594 (2019). https://doi.org/10.1145/3340555.3355719

  18. Moerland, T.M., Broekens, J., Jonker, C.M.: Emotion in reinforcement learning agents and robots: a survey. Mach. Learn. 107(2), 443–480 (2017). https://doi.org/10.1007/s10994-017-5666-0

    MathSciNet  CrossRef  Google Scholar 

  19. Zhou, Q.: Multi-layer affective computing model based on emotional psychology. Electron. Commer. Res. 18(1), 109–124 (2017). https://doi.org/10.1007/s10660-017-9265-8

    CrossRef  Google Scholar 

  20. Calvo, R.: The Oxford Handbook of Affective Computing. Oxford University Press (2015). https://doi.org/10.1093/oxfordhb/9780199942237.001.0001

  21. Broekens, J., Bosse, T., Marsella, S.C.: Challenges in computational modeling of affective processes. IEEE Trans. Affect. Comput. 4(3), 242–245 (2013). https://doi.org/10.1109/T-AFFC.2013.23

    CrossRef  Google Scholar 

  22. Taverner, J., Vivancos, E., Botti, V.: A fuzzy appraisal model for affective agents adapted to cultural environments using the pleasure and arousal dimensions. Inf. Sci. 546, 74–86 (2021). https://doi.org/10.1016/j.ins.2020.08.006

    CrossRef  Google Scholar 

  23. Ashwin, T.S., Guddeti, R.M.R.: Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Educ. Inf. Technol. 25(2), 1387–1415 (2019). https://doi.org/10.1007/s10639-019-10004-6

    CrossRef  Google Scholar 

  24. Bird, J.J., Ekárt, A., Faria, D.R.: Chatbot interaction with artificial intelligence: human data augmentation with T5 and language transformer ensemble for text classification. ArXiv: 2010.05990 [Cs] (2020). http://arxiv.org/abs/2010.05990

  25. Fiorini, L., Mancioppi, G., Semeraro, F., Fujita, H., Cavallo, F.: Unsupervised emotional state classification through physiological parameters for social robotics applications. Knowl.-Based Syst. 190, 105217 (2020). https://doi.org/10.1016/j.knosys.2019.105217

  26. Ivanovic, M., et al.: Emotional agents—state of the art and applications. Comput. Sci. Inf. Syst. (2015). https://doi.org/10.2298/CSIS141026047I

  27. Newell, A.: SOAR as a unified theory of cognition: issues and explanations. Behav. Brain Sci. 15(3), 464–492 (1992). https://doi.org/10.1017/S0140525X00069740

    CrossRef  Google Scholar 

  28. Zuo, G., Pan, T., Zhang, T., Yang, Y.: SOAR improved artificial neural network for multistep decision-making tasks. Cogn. Comput. 13(3), 612–625 (2020). https://doi.org/10.1007/s12559-020-09716-6

    CrossRef  Google Scholar 

  29. Schindler, S., Bublatzky, F.: Attention and emotion: an integrative review of emotional face processing as a function of attention. Cortex 130, 362–386 (2020). https://doi.org/10.1016/j.cortex.2020.06.010

    CrossRef  Google Scholar 

  30. Marcos, S., Gómez-García-Bermejo, J., Zalama, E.: A realistic, virtual head for human–computer interaction. Interact. Comput. 22(3), 176–192 (2010). https://doi.org/10.1016/j.intcom.2009.12.002

    CrossRef  Google Scholar 

  31. Fernandez, R., John, N., Kirmani, S., Hart, J., Sinapov, J., Stone, P.: Passive demonstrations of light-based robot signals for improved human interpretability. In: 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 234–239. https://doi.org/10.1109/ROMAN.2018.8525728

  32. MacDorman, K.F., Ishiguro, H.: The uncanny advantage of using androids in cognitive and social science research. Interact. Stud.: Soc. Behav. Commun. Biol. Artif. Syst. 7(3), 297–337 (2006). https://doi.org/10.1075/is.7.3.03mac

    CrossRef  Google Scholar 

  33. Matthews, G., et al.: Evolution and revolution: personality research for the coming world of robots, artificial intelligence, and autonomous systems. Pers. Individ. Differ. 169, 109969 (2021). https://doi.org/10.1016/j.paid.2020.109969

  34. Korn, O., Akalin, N., Gouveia, R.: Understanding cultural preferences for social robots: a study in German and Arab communities. ACM Trans. Hum.-Robot Interact. 10(2), 12:1–12:19 (2021). https://doi.org/10.1145/3439717

  35. Nishio, T., Yoshikawa, Y., Ogawa, K., Ishiguro, H.: Development of an effective information media using two android robots. Appl. Sci. 9(17), 3442 (2019). https://doi.org/10.3390/app9173442

    CrossRef  Google Scholar 

  36. Doering, M., Glas, D.F., Ishiguro, H.: Modeling interaction structure for robot imitation learning of human social behavior. IEEE Trans. Hum.-Mach. Syst. 49(3), 219–231 (2019). https://doi.org/10.1109/THMS.2019.2895753

    CrossRef  Google Scholar 

  37. Pablos, S.M., García-Bermejo, J.G., Zalama Casanova, E., López, J.: Dynamic facial emotion recognition oriented to HCI applications. Interact. Comput. 27(2), 99–119 (2015). https://doi.org/10.1093/iwc/iwt057

    CrossRef  Google Scholar 

  38. Strathearn, C., Ma, M.: Modelling user preference for embodied artificial intelligence and appearance in realistic humanoid robots. Informatics 7(3), 28 (2020). https://doi.org/10.3390/informatics7030028

    CrossRef  Google Scholar 

  39. Cañamero, L.: Emotion understanding from the perspective of autonomous robots research. Neural Netw. 18(4), 445–455 (2005). https://doi.org/10.1016/j.neunet.2005.03.003

    CrossRef  Google Scholar 

  40. Dautenhahn, K., Woods, S., Kaouri, C., Walters, M. L., Koay, K.L., Werry, I.: What is a robot companion—Friend, assistant or butler? In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1192–1197 (2005). https://doi.org/10.1109/IROS.2005.1545189

  41. Whittaker, S., Rogers, Y., Petrovskaya, E., Zhuang, H.: Designing Personas for expressive robots: personality in the new breed of moving, speaking, and colorful social home robots. ACM Trans. Hum.-Robot Interact. 10(1), 8:1–8:25 (2021). https://doi.org/10.1145/3424153

  42. Schoenherr, J.R., Burleigh, T.J.: Uncanny sociocultural categories. Front. Psychol. 5. (2015). https://doi.org/10.3389/fpsyg.2014.01456

  43. Cheetham, M.: Editorial: the uncanny valley hypothesis and beyond. Front. Psychol. 8 (2017). https://doi.org/10.3389/fpsyg.2017.01738

  44. Gee, F.C., Browne, W.N., Kawamura, K.: Uncanny valley revisited. In: ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, pp. 151–157 (2005). https://doi.org/10.1109/ROMAN.2005.1513772

  45. Brink, K.A., Gray, K., Wellman, H.M.: Creepiness creeps In: uncanny valley feelings are acquired in childhood. Child Dev. 90(4), 1202–1214. (2019). https://doi.org/10.1111/cdev.12999

  46. Feng, S., et al.: The uncanny valley effect in typically developing children and its absence in children with autism spectrum disorders. PLoS One 13(11), e0206343 (2018). https://doi.org/10.1371/journal.pone.0206343

  47. Tinwell, A., Sloan, R.J.S.: Children’s perception of uncanny human-like virtual characters. Comput. Hum. Behav. 36, 286–296 (2014). https://doi.org/10.1016/j.chb.2014.03.073

    CrossRef  Google Scholar 

Download references

Acknowledgments

This research was partially funded by the Spanish Government Ministry of Economy and Competitiveness through the DEFINES project grant number (TIN2016-80172-R) and the Ministry of Science and Innovation through the AVisSA project grant number (PID2020-118345RB-I00).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Samuel Marcos-Pablos .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Marcos-Pablos, S., García-Peñalvo, F.J. (2022). Emotional Intelligence in Robotics: A Scoping Review. In: de Paz Santana, J.F., de la Iglesia, D.H., López Rivero, A.J. (eds) New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence. DiTTEt 2021. Advances in Intelligent Systems and Computing, vol 1410. Springer, Cham. https://doi.org/10.1007/978-3-030-87687-6_7

Download citation