Abstract
One would like to rely on design guidelines for embodied conversational agents (ECAs), grounded on evaluation studies. How to define the physical and mental characteristics of an ECA, optimal for an envisioned application? What will be the added value of using an ECA? Although there have been studies addressing such issues, we are still far from getting a complete picture. This is not only due to the still relatively little experience with applications of ECAs, but also to the diversity in terms and experimental settings used. The lack of a common, established framework makes it di cult to compare ECAs, interpret evaluation results and judge their scope and relevance. In this chapter we propose a common taxonomy of the relevant design and evaluation aspects of ECAs. We refer to recent works to elicit evaluation concepts and discuss measurement issues.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
André, E. and Rist, T. (2000). Presenting Through Performing: On the Use of Multiple Lifelike Characters in Knowledge-Based Presentation Systems. In Proc. of the Second International Conference on Intelligent User Interfaces, pp. 1–8. New Orleans, Louisiana.
Bailenson, J. N., Beall, A. C., Blascovich, J., Raimundo, M. and Weisbuch, M., (2001). Intelligent Agents Who Wear Your Face: Users Reactions to the Virtual Self. In De Antonio, A., Aylett, R., Ballin, D., editors, In Proc. of the Third International Workshop Intelligent Virtual Agents, IVA 2001, pp. 86–99, Madrid, Spain, Lecture Notes in Computer Science 2190, Springer.
Barker, T. (2003). The Illusion of Life Revisited. In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Baylor, A. L. (2003). The Split-Persona E ect with Pedagogical Agents. In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Benbasat, I., Dexter, A. and Masulis, P. (1981). An Experimental Study of the Human/ComputerInterface. Communication of the ACM, 24(11): 752–762.
Bickmore, T. and, Cassell, J. (2001). A Relational Agent: A Model and Implementation of Building User Trust. In Proceedings of the CHI’01. pp. 396–403. Seattle, Washington.
Buisine, S., Abrilian, S., Rendu, C., and Martin, J.C. (2003). Evaluation of Individual Multimodal Behavior of 2D Embodied Agents in Presentation Tasks, In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Cassell, J. and Bickmore, T. (2000). External Manifestations of Trust-worthiness in the Interface. Communications of the ACM, 43(12): 50–56.
Cassell, J. Thóorisson, K. R. (1999). The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents. Applied Artificial Intelligence, 13: 519–538.
Cassell, J. and Vilhjáalmsson, H. (1999). FullyEmbodied Conversational Avatars: Making Communicative Behaviors Autonomous. Autonomous Agents and Multi-Agent Systems, 2(1): 45–64.
Cowell, A.J. and Stanney, K.M. (2003). On Manipulating Nonverbal Interaction Style to Increase Anthropomorphic Computer Character Credibility. In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Dehn, D. and Van Mulken, S. (2000). The impact of animated interface agents: a review of empirical research. Int. Journal of Human-Computer Studies. 52: 1–22.
De Rosis, F., Pelachaud, C. and Poggi, I. (to appear). Transcultural believability in embodied agents: A matter of consistent adaptation. In Payr, S. and Trappl, R., editors, Agent Culture: Designing Human-Agent Interaction in a Multicultural World. Laurence Erlbaum Associates, New York.
Dryer, D. (1999). Getting personal with computers. Applied Artificial Intelligence. 13: 273–295.
Griffin, P., Hodgson, P., and Prevost, S. (2003). Character User Interfaces for Commercial Applications. In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Hofstede, G. (1997). Cultures and Organisations: Software of the Mind. McGraw-Hill, New York.
Höök, K., Persson, P., and Sjöolinder, M. (2000). Evaluating Users Experience of a Character Enhanced Information Space, Journal of AI Communications, 13(3): 195–212.
Isbister, K. and Doyle, P. (2002). Design and Evaluation of Embodied Conversational Agents: A Proposed Taxonomy. In Proc. of the AAMAS02 Workshop on Embodied Conversational Agents: Lets Specify and Compare Them!, Bologna, Italy.
Isbister, K. and Hayes-Roth, B. (1998). Social Implications of Using Synthetic Characters: An Examination of a Role-Specific Intelligent Agent. KSL-98-01 Stanford, Knowledge Systems Laboratory.
Isbister, K., Nakanishi, H., Ishida, T., and Nass, C. (2000). Helper Agent: Designing an Assistant for Human-Human Interaction in a Virtual Meeting Space. In Proceedings of the CHI 2000, pp. 57–64. The Hague, The Netherlands.
Isbister, K. and Nass, C. (2000). Consistency of Personality in Interactive Characters: Verbal Cues, Non-Verbal Cues, and User Characteristics. International Journal of Human-Computer Studies, 53: 251–267.
Isla, D. and Blumberg, B. (2002). Object Persistence for Synthetic Creatures. In Proc. of AAMAS02, pp. 1356–1363.
King, J. and Ohya, J. (1996). The Representation of Agents: Anthropomorphism, Agency, and Intelligence. In Proc. CHI96, pp. 289–290.
Koda, T. and Maes, P. (1996). Agents With Faces: The E ects of Personification of Agents. In Proc. of HCI’96, pp. 98–103. London, UK.
Larsen, R.J. and Diener, E. (1987). Affect Intensity as an Individual Difference Characteristic: A review. Journal of Research in Personality, 21: 1–39.
Lester, J., Converse, S., Kahler, S., Barlow, S., Stone, B., and Bhogal, R. (1997). The Persona Effect: Affective Impact of Animated Pedagogical Agents. In Proc. of CHI97, pp. 359–366, ACM Press, New York.
Massaro, D.(1998). Perceiving Talking Faces. The MIT Press, Cambridge, MA, USA
McBreen, H.M., Shade, P., Jack, M.A., and Wyard, P.J. (2000). Experimental Assessment of the Effectiveness of Synthetic Personae for Multi-Modal E-Retail Applications. In Proc. of Fourth International Conference on Autonomous Agents, pp. 39–45.
McBreen, H.M., Anderson, J., and Jack, M. (2001). Evaluating 3D Embodied Conversational Agents in Contrasting VRML Retail Applications. In Proc. of AAMAS01 Workshop on Representing, Annotating, and Evaluating Non-Verbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents, Montreal, Canada.
McCraae, R. R., and John, O.P. (1992). An Introduction to the Five-Factor Model and its Applications. Journal of Personality, 60: 175–215.
Mori, J., Prendinger, H., and Ishizuka, M. (2003). Evaluation of an Embodied Conversational Agent with Affective Behavior. In Proc. of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals, Melbourne, Australia.
Moundridou, M. and Virvou, M. (2002). Evaluating the Persona Effect of an Interface Agent in an Intelligent Tutoring System. Journal of Computer Assisted Learning, 18(3): 253–261.
Nass, C., Isbister, K., and Lee, E. J. (2000). Truth is Beauty: Researching Embodied Conversational Agents. In Cassell, J., Sullivan, J., Prevost, J., Churchill, E., editors, Embodied Converastional Agents, pp. 374–401, MIT Press, MA, USA.
Nass, C. and Lee, K. M. (2000). Does Computer-Generated Speech Manifest Personality? An Experimental Test of Similarity-Attraction. In Proc. of CHI 2000, pp. 329–336. The Hague, The Netherlands.
Nielsen, J. (1993). Usability Engineering. Morgan Kaufmann, San Francisco.
Paradiso, A. and ĽAbbate, M.A. (2001). A Model for the Generation and Combination of Emotional Expressions. In Proc. of the AAMAS01 Workshop on Representing, Annotating and Evaluating Non-Verbal and Verbal Communicative Acts to Achieve Contextual Embodied Agents. Montreal, Canada.
Pelachaud, C., Carofiglio, V., De Carolis, B., de Rosis, F., and Poggi, I. (2002). Embodied Contextual Agent in Information Delivering Application. In Proc. of First International Joint Conference on Autonomous Agents and Multi-Agent Systems, Bologna, Italy.
Pelachaud, C. and Poggi, I. (2002). Subtleties of Facial Expressions in Embodied Agents. Journal of Visualization and Computer Animation, 13: 301–312.
Prendinger, H., and Ishizuka, M. (2002). Social Role Awareness in Animated Agents. In Proc. of AAMAS02, pp. 270–277.
Prendinger, H. and Ishizuka, M. (2003). Designing and Evaluating Animated Agents as Social Actors. IEICE Transactions on Information and Systems. E86-D(8): 1378–1385.
Reeves, B. and Nass, C. (1996). The Media Equation — How People Treat Computers, Television and New Media Like Real People and Places. Cambridge University Press, Cambridge.
Resnick, P.V. and Lammers, H.B. (1985). The Influence of Self-esteem on Cognitive Responses to Machine-Like Versus Human-Like Computer Feedback. The Journal of Social Psychology, 125: 761–769.
Rickenberg, R. and Reeves, B. (2000). The Effects of Animated Characters on Anxiety, Task Performance, and Evaluations of User Interfaces. In Proc. of CHI 2000, pp. 49–56. The Hague, The Netherlands.
Russell, J.A. and Mehrabian, A. (1977). Evidence for a Three-Factor Theory of Emotions. Journal of Research in Personality, 11: 273–294.
Sanders, G. and Scholtz, J. (2000). Measurement and Evaluation of Embodied Conversational Agents. In Cassell, J., Sullivan, J., Prevost, J., Churchill, E., editors, Embodied Converastional Agents, MIT Press, pp. 347–373.
Sproull, L., Subramani, M., Kiesler, S., Walker, J.H., and Waters, K. (1996). When the Interface is a Face. Human-Computer Interaction, 11: 97–124.
Van Mulken, S., André, E., Müller, J. (1998). The Persona Effect: How substantial is it?. In Proc. of HCI 1998, pp. 53–66. Sheffield, UK.
Witkowski, M., Arafa, Y., and De Bruijn, O. (2001). Evaluating User Reaction to Character Agent Mediated Displays Using Eye-Tracking Technology. In Proc. of the Workshop on Information Agents in E-commerce; Agents and Cognition, AISB Convention, York, UK.
Xiao, J., Stasko, J., and Catrambone, R. (2002). Embodied Conversational Agents as a UI Paradigm: A Framework for Evaluation. In Proc. of the AAMAS02 Workshop on Embodied Conversational Agents: Lets Specify and Compare Them!, Bologna, Italy.
Editor information
Rights and permissions
Copyright information
© 2004 Kluwer Academic Publishers
About this chapter
Cite this chapter
Ruttkay, Z., Dormann, C., Noot, H. (2004). Embodied Conversational Agents on a Common Ground. In: Ruttkay, Z., Pelachaud, C. (eds) From Brows to Trust. Human-Computer Interaction Series, vol 7. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2730-3_2
Download citation
DOI: https://doi.org/10.1007/1-4020-2730-3_2
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-2729-1
Online ISBN: 978-1-4020-2730-7
eBook Packages: Computer ScienceComputer Science (R0)