Abstract
Three experiments were conducted to assess the relative importance of speech parameters and facial expressions in the delivery of feedback by a pedagogical agent. In Experiment 1, we manipulated linguistic form (i.e., positive, negative, or neutral terms), rate, pitch, pause, and emphasis. In Experiment 2, we manipulated eye size, mouth curve, brow height, and brow curve. In a third study, both speech parameters and facial expressions were manipulated. In all three experiments, the participants were asked to indicate how positive or negative the agent's feedback seemed to be. Across the studies, the variables collectively accounted for a significant amount of the variance. More specifically, the linguistic expressions and mouth curve emerged as significant predictors of the participants' ratings. This suggests that these two features should be implemented by developers wishing to provide appropriate feedback in their pedagogical agents.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
André, E., Rist, T., and Müller, J. (1998). Integrating reactive and scripted behaviors in a life-like presentation agent. In Proceedings of the Second International Conference on Autonomous Agents. New York: ACM, pp. 261-268.
Brown, P. and Levinson, S. (1987). Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press.
Cañamero, L.D. and Fredslund, J. (2000). In K. Dautenhahn (Ed.), Socially Intelligent Agents: The Human in the Loop (Papers from the AAAI Fall Symposium). Menlo Park, CA: AAAI Press.
Cassell, J., Pelechaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., and Stone, M. (1994). Animated conversation: Rule-based generation of facial expression, gesture, and spoken intonation for multiple conversational agents. In '94).New York: ACM, pp. 413-420.
Cassell, J., Sullivan, J., Prevost, S., and Churchill, E. (Eds.). (2000). Embodied Conversational Agents. Cambridge, MA: MIT Press.
Cassell, J. and Thórisson, K.R. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, 13:519-538.
Crystal, D. (1975). The English Tone of Voice. Bristol, UK: Edward
Arnold.Davitz, J.R. (1964). Personality, perceptual, and cognitive correlates of emotional sensitivity. In J.R. Davitz (Ed.) The Communication of Emotional Meaning. New York: McGraw-Hill, pp. 57-68.
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48:384-392.
Ekman, P. and Friesen, W.F. (1984). Unmasking the Face: A Guide to Recognizing Emotions fromFacial Expressions. Englewood Cliffs, NJ: Prentice-Hall.
Ellison, J.W. and Massaro, D.W. (1997). Featural evaluation, integration, and judgment of facial affect. Journal of Experimental Psychology: Human Perception and Performance, 23:213-226.
Fox, B. (1993). The Human Tutorial Dialog Project. Hillsdale, NJ: Erlbaum.
Frick, R.W. (1985). Communicating emotion: The role of prosodic features. Psychological Bulletin,97:412-429.
Graesser, A.C., Person, N.K., and Magliano, J.P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9:1-28.
Graesser, A.C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N., and the Tutoring Research Group (2000). Usinglatent semantic analysis to evaluate the contributions of students in AutoTutor. Interactive Learning Environments, 8:129-147.
Graesser, A.C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., and the Tutoring Research Group (1999). AutoTutor: A simulation of a human tutor. Journal of Cognitive Systems Research,1:35-51.
Johnson, W.F., Emde, R.N., Scherer, K.R., and Klinnert, M.D. (1986). Recognition of emotion from vocal cues. Archives of General Psychiatry, 43:280-283.
Johnson, W.L., Rickel, J.W., and Lester, J.C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education, 11:47-78.
Landauer, T.K. and Dumais, S.T. (1997). A solution to Plato's problem: The latent semantic analysis of acquisition, induction, and representation of knowledge. Psychological Review, 104:211-240.
Lester, J., Towns, S., and FitzGerald, P. (1999). Achieving affective impact: Visual emotive communication in lifelike pedagogical agents. The International Journal of Artificial Intelligence in Education, 10:278-291.
Massaro, D.W. (1998). Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. Cambridge, MA: MIT Press.
Massaro, D.W. and Egan, P.B. (1996). Perceiving affect from the voice and the face. Psychonomic Bulletin and Review, 3:215-221.
Murray, I.R. and Arnott, J.L. (1993). Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. Journal of the Acoustical Society of America, 93:1097-1108.
Ochsman, R.B. and Chapanis, A. (1974). The effects of 10 communication modes on the behavior of teams during cooperative problem solving. International Journal of Man-Machine Studies, 6:579-619.
Paiva, A. and Machado, I. (1998). Vincent, an autonomous pedagogical agent for on-the-job training. In Proceedings of the Fourth International Conference on Intelligent Tutoring Systems. Berlin: Springer-Verlag, pp. 584-593.
Person, N.K., Bautista, L., Kreuz, R.J., Graesser, A.C., and the Tutoring Research Group (2000). The dialog advancer network: A conversation manager for AutoTutor. In ITS 2000 Proceedings of the Workshop on Modeling Human Teaching Tactics and Strategies. Montréal: Université de Montréal, pp. 86-92. Perception of Feedback and Pedagogical Agents 153
Person, N.K., Kreuz, R.J., Zwaan, R.A., and Graesser, A.C. (1995). Pragmatics and pedagogy: Conversational rules and politeness strategies may inhibit effective tutoring. Cognition and Instruction, 13:161-188.
Scherer, K.R. (1986). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99:143-165.
Scherer, K.R. and Scherer, U. (1981). Speech behaviour and personality.In J.K. Darby (Ed.), Speech Evaluation in Psychiatry. New York: Grune and Stratton, pp. 115-135.
Trower, P., Bryant, B., and Argyle, M. (1978). Social Skills and Mental Health. London: Methuen.
TruVoice 1.5 [Computer software]. (1997). Burlington, MA: Lernout and Hauspie Speech Products.
Wiemer-Hastings, P., Wiemer-Hastings, K., and Graesser, A. (1999).Improving an intelligent tutor's comprehension of students with Latent Semantic Analysis. In Artificial Intelligence in Education. Amsterdam: IOS Press, pp. 535-542.
Wierzbicka, A. (2000). The semantics of facial expressions. Pragmatics and Cognition, 8:147-184.
Author information
Authors and Affiliations
Consortia
Rights and permissions
About this article
Cite this article
Link, K.E., Kreuz, R.J., Graesser, A.C. et al. Factors that Influence the Perception of Feedback Delivered by a Pedagogical Agent. International Journal of Speech Technology 4, 145–153 (2001). https://doi.org/10.1023/A:1017383528041
Issue Date:
DOI: https://doi.org/10.1023/A:1017383528041