Abstract
Using emotional expressions is an effective dialogue technique in human–human dialogue. Introducing such techniques to human–robot interaction might improve their effectiveness to encourage the cooperative dialogue manner of system users. However, most of the existing research on emotional agent systems was based on the Wizard-of-Oz (WOZ) method to verify the abilities of interactive interfaces. In this paper, we build an autonomous dialogue robot that uses emotional expressions for eliciting the cooperative dialogue manner of users. The robot uses both verbal and multimodal expressions as well as emotional speech and emotional gestures in interactions. Our dialogue experiments showed that positive emotional expressions are the most efficient strategy for facilitating cooperative dialogues with users. Moreover, using negative emotional expressions is also an effective strategy in some dialogue contexts. We also investigated several modalities to emphasize the robot’s emotional expression abilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adler RF, Iacobelli F, Gutstein Y (2016) Are you convinced? A wizard of OZ study to test emotional vs. rational persuasion strategies in dialogues. Comput Hum Behav 57:75–81
Asai S, Yoshino K, Shinagawa S, Sakti S, Nakamura S (2020) Emotional speech corpus for persuasive dialogue system. In: Proceedings of The 12th language resources and evaluation conference, pp 491–497. Marseille, France
Becker C, Kopp S, Wachsmuth I (2004) Simulating the emotion dynamics of a multimodal conversational agent. In: Tutorial and research workshop on affective dialogue systems. Springer, pp 154–165
Colombo P, Witon W, Modi A, Kennedy J, Kapadia M (2019) Affect-driven dialog generation. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 3734–3743
bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp 4171–4186 (2019)
Fogg B (1997) Captology: the study of computers as persuasive technologies. In: Proceedings CHI extended abstracts on HFCS, CHI EA ’97, p 129
Forgas JP (1998) On feeling good and getting your way: mood effects on negotiator cognition and bargaining strategies
Galley M, Brockett C, Gao X, Dolan B, Gao J (2019) End-to-end conversation modeling: moving beyond chitchat. In: AAAI the seventh dialogue system technology challenge
Ghosh S, Chollet M, Laksana E, Morency LP, Scherer S (2017) Affect-lm: a neural language model for customizable affective text generation. In: ACL
Goswamy T, Singh I, Barkati A, Modi A (2020) Adapting a language model for controlled affective text generation. In: Proceedings of the 28th international conference on computational linguistics, pp 2787–2801
Gunasekara C, Kummerfeld JK, Polymenakos L, Lasecki W (2019) Dstc7 task 1: noetic end-to-end response selection. In: Proceedings the first workshop on NLP for conversational AI, pp 60–67
Harris ZS (1954) Distributional structure. Word 10(2–3):146–162
Heath R, Brandt D, Nairn A (2006) Brand relationships: strengthened by emotion, weakened by attention. J Advert Res 46(4):410–419
Hiraoka T, Neubig G, Sakti S, Toda T, Nakamura S (2016) Learning cooperative persuasive dialogue policies using framing. Speech Commun 84:83–96
Lhommet M, Marsella S (2014) Expressing emotion through posture. The Oxford handbook of affective computing, pp 273–285
Lorenzo-Trueba J, Barra-Chicote R, San-Segundo R, Ferreiros J, Yamagishi J, Montero J (2015) Emotion transplantation through adaptation in hmm-based speech synthesis. Comput Speech Lang 34(1):292–307. https://doi.org/10.1016/j.csl.2015.03.008
Lubis N, Sakti S, Yoshino K, Nakamura S (2018) Eliciting positive emotion through affect-sensitive dialogue response generation: a neural network approach. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Mehrabian A, Russell JA (1974) The basic emotional impact of environments. Percept Motor Skills 38(1):283–301
Mizukami M, Kizuki H, Nomura T, Neubig G, Yoshino K, Sakti S, Toda T, Nakamura S (2015) Adaptive selection from multiple response candidates in example-based dialogue. In: 2015 IEEE workshop on automatic speech recognition and understanding, pp 784–790
Morris M, Keltner D (2000) How emotions work: the social functions of emotional expression in negotiations. Res Organ Behav 22:1–50
Sakaki T, Mizuki S, Gunji N (2019) Bert pre-trained model trained on large-scale Japanese social media corpus. Hottolink
Santhanam S, Shaikh S (2019) Emotional neural language generation grounded in situational contexts. In: Proceedings the 4th workshop on computational creativity in language generation, pp 22–27
Serban IV, Sordoni A, Lowe R, Charlin L, Pineau J, Courville A, Bengio Y (2017) A hierarchical latent variable encoder-decoder model for generating dialogues. In: Thirty-First AAAI conference on artificial intelligence
Sinaceur M, Tiedens LZ (2006) Get mad and get more than even: when and why anger expression is effective in negotiations
Tuyen NTV, Jeong S, Chong NY (2018) Emotional bodily expressions for culturally competent robots through long term human-robot interaction. In: 2018 IEEE/RSJ international conference on intelligent robots and systems, pp 2008–2013
Wang X, Shi W, Kim R, Oh Y, Yang S, Zhang J, Yu Z (2019) Persuasion for good: towards a personalized persuasive dialogue system for social good. In: Proceedings of ACL
Watanabe M, Ogawa K, Ishiguro H (2018) At the department store—can androids be a social entity in the real world? In: Geminoid studies, pp 423–427
Wilson E (2003) Perceived effectiveness of interpersonal persuasion strategies in computer-mediated communication. Comput Hum Behav 19(5):537–552
Yoshino K, Ishikawa Y, Mizukami M, Suzuki Y, Sakti S, Nakamura S (2018) Dialogue scenario collection of persuasive dialogue with emotional expressions via crowdsourcing. In: Proceedings of the 11th language resources and evaluation conference
Zhou H, Huang M, Zhang T, Zhu X, Liu B (2017) Emotional chatting machine: emotional conversation generation with internal and external memory. In: AAAI
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Asai, S., Yoshino, K., Shinagawa, S., Sakti, S., Nakamura, S. (2022). Eliciting Cooperative Persuasive Dialogue by Multimodal Emotional Robot. In: Stoyanchev, S., Ultes, S., Li, H. (eds) Conversational AI for Natural Human-Centric Interaction. Lecture Notes in Electrical Engineering, vol 943. Springer, Singapore. https://doi.org/10.1007/978-981-19-5538-9_10
Download citation
DOI: https://doi.org/10.1007/978-981-19-5538-9_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-5537-2
Online ISBN: 978-981-19-5538-9
eBook Packages: Computer ScienceComputer Science (R0)