Advertisement

Fostering User Engagement in Face-to-Face Human-Agent Interactions: A Survey

Chapter
First Online:
Part of the Intelligent Systems Reference Library book series (ISRL, volume 106)

Abstract

Embodied conversational agents are capable of carrying a face-to-face interaction with users. Their use is substantially increasing in numerous applications ranging from tutoring systems to ambient assisted living. In such applications, one of the main challenges is to keep the user engaged in the interaction with the agent. The present chapter provides an overview of the scientific issues underlying the engagement paradigm, including a review on methodologies for assessing user engagement in human-agent interaction. It presents three studies that have been conducted within the Greta/VIB platforms. These studies aimed at designing engaging agents using different interaction strategies (alignment and dynamical coupling) and the expression of interpersonal attitudes in multi-party interactions.

Keywords

Embodied Conversational Agent Interaction strategies Socio-emotional behavior User engagement 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

The authors would like to thank the GRETA team for its contributions to the Greta and Vib platforms. This work has been supported by the French collaborative project A1:1, the european project ARIA-VALUSPA, and performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under reference ANR-11-IDEX-0004-02.

References

  1. 1.
    Acosta JC, Ward NG (2011) Achieving rapport with turn-by-turn, user-responsive emotional coloring. Speech Commun 53(910):1137–1148CrossRefGoogle Scholar
  2. 2.
    Adelswärd V (1989) Laughter and dialogue: the social significance of laughter in institutional discourse. Nordic J Linguist 12(02):107–136CrossRefGoogle Scholar
  3. 3.
    Albrecht I, Schröder M, Haber J, Seidel H (2005) Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Real 8(4):201–212CrossRefGoogle Scholar
  4. 4.
    Andre E, Rehm M, Minker W, Bühler D (2004) Endowing spoken language dialogue systems with emotional intelligence. Affective dialogue systems. Springer, Berlin, pp 178–187Google Scholar
  5. 5.
    Argyle M (1988) Bodily communication, 2nd edn. Methuen, New YorkGoogle Scholar
  6. 6.
    Bazzanella C (2011) Redundancy, repetition, and intensity in discourse. Lang Sci 33(2):243–254MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bee N, André E, Tober S (2009) Breaking the ice in human-agent communication: eye-gaze based initiation of contact with an embodied conversational agent. In: Proceedings of 9th international conference on Intelligent virtual agents, IVA, Amsterdam, The Netherlands, Sept 14–16, 2009, pp 229–242Google Scholar
  8. 8.
    Bee N, Pollock C, André E, Walker M (2010) Bossy or wimpy: expressing social dominance by combining gaze and linguistic behaviors. In: Proceedings of the 10th international conference on intelligent virtual agents, Springer, Berlin, Heidelberg, IVA’10, pp 265–271Google Scholar
  9. 9.
    Bergmann K, Eyssel F, Kopp S (2012) A second chance to make a first impression? how appearance and nonverbal behavior affect perceived warmth and competence of virtual agents over time. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture Notes in Computer Science, vol 7502, Springer, Berlin Heidelberg, pp 126–138, doi: 10.1007/978-3-642-33197-8_13 Google Scholar
  10. 10.
    Bertrand R, Ferré G, Guardiola M et al (2013) French face-to-face interaction: repetition as a multimodal resource. Interaction, Coverbal Synchrony in Human-Machine, p 141Google Scholar
  11. 11.
    Beskow J (1997) Animation of talking agents. In: Benoit C, Campbell R (eds) Proceedings of the ESCA workshop on audio-visual speech processing. Rhodes, Greece, pp 149–152Google Scholar
  12. 12.
    Bickmore T, Giorgino T (2006) Health dialog systems for patients and consumers. J biomed inf 39(5):556–571CrossRefGoogle Scholar
  13. 13.
    Bickmore T, Gruber A, Picard R (2005) Establishing the computer patient working alliance in automated health behavior change interventions. Patient Educ Couns 59(1):21–30. doi: 10.1016/j.pec.2004.09.008 CrossRefGoogle Scholar
  14. 14.
    Bickmore T, Schulman D, Yin L (2010) Maintaining engagement in long-term interventions with relational agents. Appl Artif Intell 24(6):648–666CrossRefGoogle Scholar
  15. 15.
    Bickmore T, Pfeifer L, Schulman D (2011) Relational agents improve engagement and learning in science museum visitors. Intelligent virtual agents. Springer, Berlin, pp 55–67Google Scholar
  16. 16.
    Bickmore TW, Picard RW (2005) Establishing and maintaining long-term human-computer relationships. ACM Trans Comput-Hum Interact 12(2):293–327CrossRefGoogle Scholar
  17. 17.
    Bohus D, Horvitz E (2014) Managing human-robot engagement with forecasts and... um... hesitations. In: Proceedings of the 16th international conference on multimodal interaction, ACM, pp 2–9Google Scholar
  18. 18.
    Boukricha H, Wachsmuth I, Hofstätter A, Grammer K (2009) Pleasure-arousal-dominance driven facial expression simulation. In: Proceedings of third international conference and workshops on Affective computing and intelligent interaction, ACII, Amsterdam, The Netherlands, Sept 10–12 2009, pp 1–7Google Scholar
  19. 19.
    Branigan HP, Pickering MJ, Cleland AA (2000) Syntactic co-ordination in dialogue. Cognition 75(2):B13–B25CrossRefGoogle Scholar
  20. 20.
    Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in conversation. J Exp Psychol: Learn, Mem, Cognit 22(6):1482Google Scholar
  21. 21.
    Bui TD, Heylen D, Poel M, Nijholt A (2001) Generation of facial expressions from emotion using a fuzzy rule based system. In: Stumptner M, Corbett D, Brooks M (eds) Proceedings of 14th Australian joint conference on artificial intelligence (AI 2001). Springer, Adelaide, Australia, pp 83–94Google Scholar
  22. 22.
    Burgoon J, Buller D, Hale J, de Turck M (1984) Relational messages associated with nonverbal behaviors. Hum Commun Res 10(3):351–378CrossRefGoogle Scholar
  23. 23.
    Buschmeier H, Bergmann K, Kopp S (2009) An alignment-capable microplanner for natural language generation. In: Proceedings of the 12th European workshop on natural language generation, association for computational linguistics, pp 82–89Google Scholar
  24. 24.
    Cafaro A, Vilhjálmsson HH (2015) First impressions in human-agent virtual encounters. ACM Trans Comput-Hum Interact (TOCHI, forthcoming)Google Scholar
  25. 25.
    Cafaro A, Vilhjálmsson HH, Bickmore TW, Heylen D, Schulman D (2013) First impressions in user-agent encounters: the impact of an agent’s nonverbal behavior on users’ relational decisions. In: Proceedings of the 2013 international conference on autonomous agents and multi-agent systems, international foundation for autonomous agents and multiagent systems, Richland, SC, AAMAS ’13, pp 1201–1202. http://dl.acm.org/citation.cfm?id=2484920.2485142
  26. 26.
    Cafaro A, Vilhjlmsson H, Bickmore T, Heylen D, Jhannsdttir K, Valgarsson G (2012) First impressions: users judgments of virtual agents personality and interpersonal attitude in first encounters. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Springer, Berlin Heidelberg, Richland, SC, pp 67–80. doi: 10.1007/978-3-642-33197-8_7 Google Scholar
  27. 27.
    Callejas Z, Griol D (2011) López-Cózar R (2011) Predicting user mental states in spoken dialogue systems. EURASIP J Adv Signal Proc 1:6CrossRefGoogle Scholar
  28. 28.
    Campano S, Sabouret N (2009) A socio-emotional model of impoliteness for non-player characters. In: 3rd international conference on affective computing and intelligent interaction and workshops. ACII 2009. IEEE, pp 1–7Google Scholar
  29. 29.
    Campano S, Durand J, Clavel C (2014) Comparative analysis of verbal alignment in human-human and human-agent interactions. In: Proceedings of the 9th international conference on language resources and evaluation (LREC’14), European language resources association (ELRA)Google Scholar
  30. 30.
    Campano S, Langlet C, Glas N, Clavel C, Pelachaud C (2015) An eca expressing appreciations. In: First international workshop on engagement in human computer interaction ENHANCE’15 held in conjuction with the 6th international conference on affective computing and intelligent interaction (ACII 2015), IEEE, sept 2015 (to appear)Google Scholar
  31. 31.
    Campano S, Clavel C, Pelachaud C (2015) I like this painting too: when an eca shares appreciations to engage users. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systemsGoogle Scholar
  32. 32.
    Cassell J (2000) Embodied conversational agents. MIT press, CambridgeGoogle Scholar
  33. 33.
    Castellano G, Mancini M, Peters C, McOwan PW (2012) Expressive copying behavior for social agents: a perceptual analysis. IEEE Trans Syst, Man Cybern, Part A: Syst Hum 42(3):776–783CrossRefGoogle Scholar
  34. 34.
    Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J pers soc psychol 76(6):893CrossRefGoogle Scholar
  35. 35.
    Choi A, Melo CD, Woo W, Gratch J (2012) Affective engagement to emotional facial expressions of embodied social agents in a decision-making game. Comput Animat Virtual Worlds 23(3–4):331–342Google Scholar
  36. 36.
    Chollet M, Ochs M, Clavel C, Pelachaud C, (2013) A multimodal corpus approach to the design of virtual recruiters. In, (2013) humaine association conference on affective computing and intelligent interaction, ACII 2013. Geneva, Switzerland, Sept 2–5:19–24Google Scholar
  37. 37.
    Clavel C, Callejas Z (2015) Sentiment analysis: from opinion mining to human-agent interaction. IEEE Trans Affect Comput 99:1. doi: 10.1109/TAFFC.2015.2444846 CrossRefGoogle Scholar
  38. 38.
    Clavel C, Vasilescu I, Devillers L, Richard G, Ehrette T (2008) Fear-type emotions recognition for future audio-based surveillance systems. Speech Commun 50:487–503Google Scholar
  39. 39.
    Courgeon M, Clavel C, Martin J (2014) Modeling facial signs of appraisal during interaction: impact on users’ perception and behavior. In: International conference on autonomous agents and multi-agent systems, AAMAS ’14, Paris, France, May 5–9 2014, pp 765–772Google Scholar
  40. 40.
    Creswell JW (2013) Research design: Qualitative, quantitative, and mixed methods approaches (Sage publications)Google Scholar
  41. 41.
    De Carolis B, Pelachaud C, Poggi I, Steedman M (2004) Apml, a markup language for believable behavior generation. In: Prendinger H, Ishizuka M (eds) Life-like characters, cognitive technologies, Springer, Berlin Heidelberg, pp 65–85. doi: 10.1007/978-3-662-08373-4_4 Google Scholar
  42. 42.
    Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D (2012) Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans Affect Comput 3(3):349–365CrossRefGoogle Scholar
  43. 43.
    Dias J, Paiva A (2005) Feeling and reasoning: a computational model for emotional agents. In: Proceedings of 12th Portuguese conference on artificial intelligence, EPIA 2005, Springer, pp 127–140Google Scholar
  44. 44.
    Ding Y, Prepin K, Huang J, Pelachaud C, Artières T (2014) Laughter animation synthesis. In: Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, AAMAS ’14, pp 773–780Google Scholar
  45. 45.
    D’Mello S, Graesser A (2013) AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Trans Interact Intell Syst 2(4):1–39CrossRefGoogle Scholar
  46. 46.
    D’Mello S, Olney A, Williams C, Hays P (2012) Gaze tutor: a gaze-reactive intelligent tutoring system. Int J hum-comput stud 70(5):377–398CrossRefGoogle Scholar
  47. 47.
    Dybala P, Ptaszynski M, Rzepka R, Araki K (2009) Activating humans with humor - a dialogue system that users want to interact with. IEICE Trans Inf Syst E92-D(12):2394–2401Google Scholar
  48. 48.
    Ekman P (2003) Emotions revealed. Times Books (US), London, New York. Weidenfeld and Nicolson (world)Google Scholar
  49. 49.
    Ekman P, Friesen W (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall Inc, Englewood CliffsGoogle Scholar
  50. 50.
    Ekman P, Friesen W, Hager J (2002) The facial action coding system, 2nd edn. Research Nexus eBook, London, Salt Lake City, Weidenfeld and Nicolson (world)Google Scholar
  51. 51.
    Fontaine JR, Scherer KR, Roesch EB, Ellsworth P (2007) The world of emotion is not two-dimensional. Psychol Sci 13:1050–1057CrossRefGoogle Scholar
  52. 52.
    Fukayama A, Ohno T, Mukawa N, Sawaki M, Hagita N (2002) Messages embedded in gaze of interface agents–impression management with agent’s gaze. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, NY, USA, CHI ’02, pp 41–48. doi: 10.1145/503376.503385
  53. 53.
    Garrod S, Anderson A (1987) Saying what you mean in dialogue: a study in conceptual and semantic co-ordination. Cognition 27(2):181–218CrossRefGoogle Scholar
  54. 54.
    Giles H, Coupland N, Coupland J (1991) Contexts of accommodation: developments in applied sociolinguistics, Cambridge, Cambridge University Press, chap 1–Accommodation theory: communication context, and consequenceGoogle Scholar
  55. 55.
    Gillies M, Ballin D (2003) A model of interpersonal attitude and posture generation. In: Proceedings of 4th international workshop on Intelligent agents, IVA 2003, Kloster Irsee, Germany, Sept 15–17 2003, pp 88–92Google Scholar
  56. 56.
    Gillies M, Ballin D (2004) Integrating autonomous behavior and user control for believable agents. In: 3rd International joint conference on autonomous agents and multiagent systems AAMAS, New York, NY, USA, August 19–23 2004, pp 336–343Google Scholar
  57. 57.
    Glas N, Pelachaud C (2014) Politeness versus perceived engagement: an experimental study. In: Proceedings of the 11th international workshop on natural language processing and cognitive scienceGoogle Scholar
  58. 58.
    Gockley R, Bruce A, Forlizzi J, Michalowski M, Mundell A, Rosenthal S, Sellner B, Simmons R, Snipes K, Schultz A, Wang J (2005) Designing robots for long-term social interaction. In: IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 1338–1343. doi: 10.1109/IROS.2005.1545303
  59. 59.
    Grammer K, Oberzaucher E (2006) The reconstruction of facial expressions in embodied systems. ZiF: mitteilungen, vol 2Google Scholar
  60. 60.
    Gratch J, Kang SH, Wang N (2013) Using social agents to explore theories of rapport and emotional resonance. Oxford University Press, Oxford, Social emotions in nature and artifact, p 181Google Scholar
  61. 61.
    Griol D, Molina JM, Callejas Z (2014) Modeling the user state for context-aware spoken interaction in ambient assisted living. Appl Intell 40:1–23CrossRefGoogle Scholar
  62. 62.
    Hall ET (1966) The hidden dimension. Doubleday, Garden CityGoogle Scholar
  63. 63.
    Hall L, Woods S, Aylett R, Newall L, Paiva A (2005) Achieving empathic engagement through affective interaction with synthetic characters. Affective computing and intelligent interaction. Springer, Heidelberg, pp 731–738Google Scholar
  64. 64.
    Hess U, Philippot P, Blairy S (1999) The social context of nonverbal behavior mimicry. cambridge University Press, Cambridge, p 213Google Scholar
  65. 65.
    Heylen D, Kopp S, Marsella SC, Pelachaud C, Vilhjálmsson HH (2008) The next step towards a function markup language. Proceedings of the 8th international conference on Intelligent Virtual Agents. Springer, Berlin, IVA, pp 270–280Google Scholar
  66. 66.
    Huber T, Ruch W (2007) Laughter as a uniform category? a historic analysis of different types of laughter. In: 10th Congress of the Swiss Society of Psychology. University of Zurich, SwitzerlandGoogle Scholar
  67. 67.
    Ivaldi S, Anzalone SM, Rousseau W, Sigaud O, Chetouani M (2014) Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement. Front Neurorobot 8:5CrossRefGoogle Scholar
  68. 68.
    Izard C (1994) Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull 115:288–299CrossRefGoogle Scholar
  69. 69.
    Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters (Studies in interactional sociolinguistics). Cambridge University Press, New YorkGoogle Scholar
  70. 70.
    Kopp S, Krenn B, Marsella S, Marshall AN, Pelachaud C, Pirker H, Thórisson KR, Vilhjálmsson HH (2006) Towards a common framework for multimodal generation: the behavior markup language. Proceedings of the 6th international conference on Intelligent virtual agents. Springer, Heidelberg, IVA, pp 205–217Google Scholar
  71. 71.
    Kopp S (2010) Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Commun 52(6):587–597CrossRefGoogle Scholar
  72. 72.
    Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum guide design and evaluation of a real-world application. In: Panayiotopoulos T, Gratch J, Aylett R, Ballin D, Olivier P, Rist T (eds) Intelligent virtual agents, Lecture notes in computer science, vol 3661, Springer, Berlin, pp 329–343. doi: 10.1007/11550617_28 Google Scholar
  73. 73.
    Lambertz K (2011) Back-channelling: The use of yeah and mm to portray engaged listenership. Griffith working papers in pragmatics and intercultural communication. vol 4, pp 11–18Google Scholar
  74. 74.
    Langlet C, Clavel C (2014) Modelling user’s attitudinal reactions to the agent utterances: focus on the verbal content. In: Proceedings of 5th international workshop on corpora for research on emotion, sentiment & social signals (ES3), Reykjavik, IcelandGoogle Scholar
  75. 75.
    Langlet C, Clavel C (2015) Improving social relationships in face-to-face human-agent interactions: when the agent wants to know users likes and dislikes. In: Proceedings of annual meeting of the association for computational linguisticGoogle Scholar
  76. 76.
    Lee J, Marsella S (2006) Nonverbal behavior generator for embodied conversational agents. In: Gratch J, Young M, Aylett R, Ballin D, Olivier P (eds), Proceedings 6th international conference intelligent virtual agents (IVA) Marina Del Rey, CA Springer, LNCS, vol 4133, Aug 21–23 2006, pp 243–255Google Scholar
  77. 77.
    Litman DJ, Forbes-Riley K (2006) Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun 48(5):559–590CrossRefGoogle Scholar
  78. 78.
    Lombard M, Weinstein L, Ditton T (2011) Measuring telepresence: the validity of the temple presence inventory (tpi) in a gaming context. In: Proceedings of 2011 Annual conference of the International Society for Presence Research (ISPR), ISPR 2011Google Scholar
  79. 79.
    Maat M, Heylen D (2009) Turn management or impression management? In: Proceedings of the 9th international conference on intelligent virtual agents, Springer, Heidelberg, IVA, pp 467–473. doi: 10.1007/978-3-642-04380-2_51 Google Scholar
  80. 80.
    Maat MT, Truong KP, Heylen D (2010) How turn-taking strategies influence users’ impressions of an agent. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Springer, Heidelberg, Lecture notes in computer science, vol 6356, pp 441–453. doi: 10.1007/978-3-642-15892-6_48 Google Scholar
  81. 81.
    Magnusson M (2000) Discovering hidden time patterns in behavior: T-patterns and their detection. Behav Res Meth, Instrum Comput 32(1):93–110CrossRefGoogle Scholar
  82. 82.
    Martin JR, White PR (2005) The language of evaluation. Palgrave Macmillan Basingstoke, New YorkGoogle Scholar
  83. 83.
    Martin JR, White PR (2005) The Language of Evaluation. Appraisal in English, Palgrave Macmillan Basingstoke, New YorkGoogle Scholar
  84. 84.
    Mehrabian A (1980) Basic dimensions for a general psychological theory: implications for personality, social, environmental, and developmental studies. Oelgeschlager, Gunn & Hain, CambridgeGoogle Scholar
  85. 85.
    Mol L, Krahmer E, Maes A, Swerts M (2012) Adaptation in gesture: converging hands or converging minds? J Mem Lang 66(1):249–264CrossRefGoogle Scholar
  86. 86.
    Morency LP, de Kok I, Gratch J (2010) A probabilistic multimodal approach for predicting listener backchannels. Auto Agents Multi-Agent Syst 20(1):70–84CrossRefGoogle Scholar
  87. 87.
    Munezero MD, Suero Montero C, Sutinen E, Pajunen J (2014) Are they different? affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Trans Affect Comput 5:101–111CrossRefGoogle Scholar
  88. 88.
    Nakano YI, Ishii R (2010) Estimating user’s engagement from eye-gaze behaviors in human-agent conversations. In: Proceedings of the 15th international conference on intelligent user interfaces, ACM, New York, IUI, pp 139–148. doi: 10.1145/1719970.1719990
  89. 89.
    Nicolle J, Rapp V, Bailly K, Prevost L, Chetouani M (2012) Robust continuous prediction of human emotions using multiscale dynamic cues. In: Proceedings of the 14th ACM international conference on multimodal interaction, pp 501–508Google Scholar
  90. 90.
    Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social interaction. J Lang Soc Psychol 21(4):337–360CrossRefGoogle Scholar
  91. 91.
    Niewiadomski R, Pelachaud C (2007) Model of facial expressions management for an embodied conversational agent. In: Proceedings of 2nd international conference on affective computing and intelligent interaction ACII, LisbonGoogle Scholar
  92. 92.
    Niewiadomski R, Hyniewska SJ, Pelachaud C (2011) Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Trans Affect Comput 2(3):134–146Google Scholar
  93. 93.
    Niewiadomski R, Demeure V, Pelachaud C (2010) Warmth, competence, believability and virtual agents. In: Allbeck J, Badler N, Bickmore T, Pelachaud C, Safonova A (eds) Intelligent virtual agents, Lecture notes in computer science, vol 6356, Springer, Heidelberg, pp 272–285. doi: 10.1007/978-3-642-15892-6_29 Google Scholar
  94. 94.
    Niewiadomski R, Obaid M, Bevacqua E, Looser J, Anh LQ, Pelachaud C (2011b) Cross-media agent platform. In: Proceedings of the 16th international conference on 3D web technology, ACM, New York, Web 3D, pp 11–19. doi: 10.1145/2010425.2010428
  95. 95.
    Ochs M, Prepin K, Pelachaud C (2013) From emotions to interpersonal stances: Multi-level analysis of smiling virtual characters. In: Proceedings of 2013 humaine association conference on affective computing and intelligent interaction (ACII), IEEE, pp 258–263Google Scholar
  96. 96.
    Osherenko A, Andre E, Vogt T (2009) Affect sensing in speech: Studying fusion of linguistic and acoustic features. In: Proceedings of affective computing and intelligent interaction (ACII)Google Scholar
  97. 97.
    Paleari M, Lisetti C (2006) Psychologically grounded avatars expressions. In: Proceedings of first workshop on emotion and computing at KI, 29th annual conference on artificial intelligence, Bremen, GermanyGoogle Scholar
  98. 98.
    Pardo JS (2006) On phonetic convergence during conversational interaction. J Acoust Soc Am 119(4):2382–2393CrossRefGoogle Scholar
  99. 99.
    Pecune F, Cafaro A, Chollet M, Philippe P, Pelachaud C (2014) Suggestions for extending saiba with the vib platform. In: Workshop on architectures and standards for IVAs, held at the ‘14th international conference on intelligent virtual agents (IVA)’, Bielefeld eCollections, pp 16–20Google Scholar
  100. 100.
    Pecune F, Mancini M, Biancardi B, Varni G, Volpe G, Ding Y, Pelachaud C, Camurri A (2015) Laughing with a virtual agent. In: Proceedings of the 14th international joint conference on Autonomous agents and multiagent systemsGoogle Scholar
  101. 101.
    Pedica C, Vilhjálmsson HH (2010) Spontaneous avatar behavior for human territoriality. Appl Artif Intell 24(6):575–593. doi: 10.1080/08839514.2010.492165 Google Scholar
  102. 102.
    Pekrun R, Cusack A, Murayama K, Elliot AJ, Thomas K (2014) The power of anticipated feedback: effects on students’ achievement goals and achievement emotions. Learn Instruct 29:115–124CrossRefGoogle Scholar
  103. 103.
    Pelachaud C, Badler N, Steedman M (1996) Generating facial expressions for speech. Cognit Sci 20(1):1–46CrossRefGoogle Scholar
  104. 104.
    Perrin L, Deshaies D, Paradis C (2003) Pragmatic functions of local diaphonic repetitions in conversation. J Pragmat 35(12):1843–1860CrossRefGoogle Scholar
  105. 105.
    Peters C, Castellano G, de Freitas S (2009) An exploration of user engagement in hci. In: Proceedings of the international workshop on affective-aware virtual agents and social robots, ACM, p 9Google Scholar
  106. 106.
    Pickering MJ, Garrod S (2004) Toward a mechanistic psychology of dialogue. Behav brain sci 27(02):169–190Google Scholar
  107. 107.
    Plutchnik R (1980) Emotion: a psychoevolutionary synthesis. Harper and Row, NYGoogle Scholar
  108. 108.
    Poggi I (2007) Mind, hands, face and body: a goal and belief view of multimodal communication. Weidler, BerlinGoogle Scholar
  109. 109.
    Prepin K, Ochs M, Pelachaud C (2013) Beyond backchannels: co-construction of dyadic stance by reciprocal reinforcement of smiles between virtual agents. In: International conference CogSci (Annual conference of the cognitive science society)Google Scholar
  110. 110.
    Provine RR (2001) Laughter: a scientific investigation. Penguin, New YorkGoogle Scholar
  111. 111.
    Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agents non-verbal behavior to a computational model of interpersonal attitudes. In: Aylett R, Krenn B, Pelachaud C, Shimodaira H (eds) Intelligent virtual agents, Lecture notes in computer science. Springer, Heidelberg, pp 263–274. doi: 10.1007/978-3-642-40415-3_23 CrossRefGoogle Scholar
  112. 112.
    Ravenet B, Ochs M, Pelachaud C (2013) From a user-created corpus of virtual agent’s non-verbal behavior to a computational model of interpersonal attitudes. In: Proceedings 13th international conference intelligent virtual agents, IVA 2013, Edinburgh, UK, August 29–31 2013, pp 263–274Google Scholar
  113. 113.
    Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Bickmore T, Marsella S, Sidner C (eds) Intelligent virtual agents, Lecture notes in computer science, vol 8637, Springer International Publishing, pp 345–349. doi: 10.1007/978-3-319-09767-1_45 Google Scholar
  114. 114.
    Ravenet B, Cafaro A, Ochs M, Pelachaud C (2014) Interpersonal attitude of a speaking agent in simulated group conversations. In: Proceedings of 14th international conference on intelligent virtual agents, IVA 2014, Boston, August 27–29, 2014, pp 345–349Google Scholar
  115. 115.
    Reynolds C (1999) Steering behaviors for autonomous characters. Miller Freeman Game Groups, San Francisco, Proceedings of the game developers conference, p 763Google Scholar
  116. 116.
    Rich C, Sidner CL (2012) Using collaborative discourse theory to partially automate dialogue tree authoring. In: Proceedings of the 12th international conference on intelligent virtual agents, Springer, Heidelberg, IVA’12, pp 327–340. doi: 10.1007/978-3-642-33197-8_34 Google Scholar
  117. 117.
    Russell J (1980) A circumplex model of affect. J Person Soc Psychol 39:1161–1178CrossRefGoogle Scholar
  118. 118.
    Ruttkay Z, Noot H, ten Hagen P (2003) Emotion disc and emotion squares: tools to explore the facial expression face. Comput Graph Forum 22(1):49–53CrossRefGoogle Scholar
  119. 119.
    Scherer K (2000) Emotion. In: Stroebe W, Hewstone M (eds) Introduction to social psychology, a European perspective. Oxford Blackwell Publishers, Oxford, pp 151–191Google Scholar
  120. 120.
    Scherer K (2005) What are emotions? And how can they be measured? Soc sci inf 44(4):695–729CrossRefGoogle Scholar
  121. 121.
    Schmitt A, Polzehl T, Minker W (2010) Facing reality: simulating deployment of anger recognition in IVR systems. In: Lee GG, Mariani J, Minker W, Nakamura S (eds) Spoken dialogue systems for ambient environments. Lecture notes in computer scienceSpringer, Heidelberg, pp 122–131CrossRefGoogle Scholar
  122. 122.
    Schroder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, Ter Maat M, McKeown G, Pammi S, Pantic M et al (2012) Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput 3(2):165–183CrossRefGoogle Scholar
  123. 123.
    Schuller B, Rigoll G, Lang M (2004) Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. Proceedings of ICASSP, Montreal 1:577–580Google Scholar
  124. 124.
    Schuller B, Batliner A, Steidl S, Seppi D (2011) Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun 53(9):1062–1087CrossRefGoogle Scholar
  125. 125.
    Sidner CL, Dzikovska M (2002) Human-robot interaction: engagement between humans and robots for hosting activities. In: Proceedings of the 4th IEEE international conference on multimodal interfaces, IEEE Computer Society, p 123Google Scholar
  126. 126.
    Sidner CL, Lee C, Kidd CD, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166(1):140–164CrossRefGoogle Scholar
  127. 127.
    Silverman LH (1999) Meaning making matters: communication, consequences, and exhibit design. ExhibitionistGoogle Scholar
  128. 128.
    Smith C, Crook N, Dobnik S, Charlton D (2011) Interaction strategies for an affective conversational agent. Presence: teleoperators and virtual environments. MIT Press, Cambridge, pp 395–411Google Scholar
  129. 129.
    Srikant R, Agrawal R (1996) Mining sequential patterns: generalizations and performance improvements. Adv Database Technol 1057:1–17Google Scholar
  130. 130.
    Stivers T (2008) Stance, alignment, and affiliation during storytelling: when nodding is a token of affiliation. Res Lang soc interact 41(1):31–57CrossRefGoogle Scholar
  131. 131.
    Svennevig J (2004) Other-repetition as display of hearing, understanding and emotional stance. Discourse studies 6(4):489–516CrossRefGoogle Scholar
  132. 132.
    Takashima K, Omori Y, Yoshimoto Y, Itoh Y, Kitamura Y, Kishino F (2008) Effects of avatar’s blinking animation on person impressions. In: Proceedings of graphics interface, Toronto, Ontario, Canada, GI ’08, Canadian Information Processing Society, pp 169–176. doi: 10.1145/1375714.1375744
  133. 133.
    Tannen D (1992) Talking voices: repetition, dialogue, and imagery in conversational discourse. cambridge University Press, CambridgeGoogle Scholar
  134. 134.
    Traum D, Aggarwal P, Artstein R, Foutz S, Gerten J, Katsamanis A, Leuski A, Noren D, Swartout W (2012) Ada and grace: Direct interaction with museum visitors. In: Nakano Y, Neff M, Paiva A, Walker M (eds) Intelligent virtual agents, Lecture notes in computer science, vol 7502, Berlin Heidelberg, Springer, pp 245–251. doi: 10.1007/978-3-642-33197-8_25 Google Scholar
  135. 135.
    Truong KP, Poppe R, Heylen D (2010) A rule-based backchannel prediction model using pitch and pause information, pp 3058–3061Google Scholar
  136. 136.
    Tsapatsoulis N, Raouzaiou A, Kollias S, Cowie R, Douglas-Cowie E (2002) Emotion recognition and synthesis based on MPEG-4 FAPs in MPEG-4 facial animation. In: Pandzic IS, Forcheimer R (eds) MPEG4 facial animation–the standard, implementations and applications. Wiley, New YorkGoogle Scholar
  137. 137.
    Vilhjálmsson HH, Cantelmo N, Cassell J, E Chafai N, Kipp M, Kopp S, Mancini M, Marsella S, Marshall AN, Pelachaud C, Ruttkay Z, Thórisson KR, Welbergen H, Werf RJ (2007) The behavior markup language: recent developments and challenges. In: Proceedings of the 7th international conference on intelligent virtual agents, Heidelberg, IVA ’07, Springer, pp 99–111Google Scholar
  138. 138.
    Vinciarelli A, Pantic M, Bourlard H (2009) Social signal processing: survey of an emerging domain. Image Vis Comput 27(12):1743–1759CrossRefGoogle Scholar
  139. 139.
    Wang N, Johnson WL, Mayer RE, Rizzo P, Shaw E, Collins H (2008) The politeness effect: pedagogical agents and learning outcomes. Int J Hum-Comput Studies 66(2):98–112CrossRefGoogle Scholar
  140. 140.
    Ward A, Litman D (2007) Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In: Proceedings of the SLaTE Workshop on speech and language technology in educationGoogle Scholar
  141. 141.
    With S, Kaiser S (2011) Sequential patterning of facial actions in the production and perception of emotional expressions. Swiss Journal of Psychology / Schweizerische Zeitschrift fr Psychologie / Revue Suisse de Psychologie 70(4):241–252Google Scholar
  142. 142.
    Yildirim S, Narayanan S, Potamianos A (2011) Detecting emotional state of a child in a conversational computer game. Comput Speech Lang 25(1):29–44CrossRefGoogle Scholar
  143. 143.
    Zhang L (2009) Exploration of affect sensing from speech and metaphorical text. In: learning by playing. Game-based education system design and development, springer, Berlin, pp 251–262Google Scholar
  144. 144.
    Zhao R, Papangelis A, Cassell J (2014) Towards a dyadic computational model of rapport management for human-virtual agent interaction. Intelligent virtual agents. Springer, Berlin, pp 514–527Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Telecom-ParisTechLTCI, CNRS, Université Paris-SaclayParisFrance
  2. 2.Lab Object’iveObject’iveParisFrance

Personalised recommendations

Cite chapter

Buy options