Effects of a Virtual Model’s Pitch and Speech Rate on Affective and Cognitive Learning

  • Sofia FountoukidouEmail author
  • Uwe Matzat
  • Jaap Ham
  • Cees Midden
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11433)


Various nonverbal behaviors – often referred to as nonverbal immediacy - of both human and virtual teachers have been shown to play a crucial role in student learning. However, past literature provides limited evidence of the effect of a virtual agent’s vocalization, as a nonverbal immediacy cue, on learning outcomes. Even less is known about the effect of nonverbal immediacy on learning when used in conjunction with virtual behavioral modeling. Earlier research provides evidence that virtual behavioral modeling could be an effective instructional method to facilitate learning in multimedia learning environments. The current work investigated the effects of a virtual model that used stronger vs. weaker vocal nonverbal immediacy on affective learning and cognitive learning (both perceived and objective). The operationalization of the virtual model’s vocal nonverbal immediacy has been realized by manipulating the agent’s vocal parameters of pitch and speech rate. We predicted, that a virtual model with stronger vocal nonverbal immediacy (i.e., higher pitch and faster speech rate) would be more effective in influencing individuals’ learning outcomes, as compared to a virtual model with weaker vocal nonverbal immediacy (i.e., lower pitch and slower speech rate). In accordance with our hypotheses, results revealed that participants who received instructions from a virtual model that used stronger vocal nonverbal immediacy showed greater affective learning, and increased perceptions of learning. Support was also found for an effect on participants’ recall. Results and implications of the study’s findings are discussed.


Nonverbal immediacy Vocalization Virtual modeling Learning 


  1. 1.
    André, E., et al.: Non-verbal persuasion and communication in an affective agent. In: Cowie, R., Pelachaud, C., Petta, P. (eds.) Emotion-Oriented Systems. Cognitive Technologies. Springer, Berlin (2011). Scholar
  2. 2.
    Morris, D.: Gestures, Their Origins and Distribution. Stein & Day Pub, New York (1979)Google Scholar
  3. 3.
    Knapp, M.L., Hall, J.A., Horgan, T.G.: Nonverbal Communication in Human Interaction. Cengage Learning, Boston (2013)Google Scholar
  4. 4.
    Mehrabian, A.: Silent Messages. Wadsworth, Belmont (1971)Google Scholar
  5. 5.
    Andersen, J.F.: Teacher immediacy as a predictor of teaching effectiveness. Ann. Int. Commun. Assoc. 3(1), 543–559 (1979)CrossRefGoogle Scholar
  6. 6.
    Witt, P.L., Wheeless, L.R., Allen, M.: A meta-analytical review of the relationship between teacher immediacy and student learning. Commun. Monogr. 71(2), 184–207 (2004)CrossRefGoogle Scholar
  7. 7.
    Clark, R.E., Choi, S.: Five design principles for experiments on the effects of animated pedagogical agents. J. Educ. Comput. Res. 32(3), 209–225 (2005)CrossRefGoogle Scholar
  8. 8.
    Reeves, B., Nass, C.: The Media Equation: How People Treat Computers, Television, and New Media. Cambridge University Press, Cambridge (1997)Google Scholar
  9. 9.
    Johnson, W.L., Rickel, J.W., Lester, J.C.: Animated pedagogical agents: face-to-face interaction in interactive learning environments. Int. J. Artif. Intell. Educ. 11(1), 47–78 (2000)Google Scholar
  10. 10.
    Fountoukidou, S., Ham, J., Matzat, U., Midden, C.: Using an artificial agent as a behavior model to promote assistive technology acceptance. In: Ham, J., Karapanos, E., Morita, P.P., Burns, C.M. (eds.) PERSUASIVE 2018. LNCS, vol. 10809, pp. 285–296. Springer, Cham (2018). Scholar
  11. 11.
    Bandura, A.: Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev. 84(2), 191 (1977)CrossRefGoogle Scholar
  12. 12.
    Rosenberg-Kima, R.B., Baylor, A.L., Plant, E.A., Doerr, C.E.: Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput. Hum. Behav. 24(6), 2741–2756 (2008)CrossRefGoogle Scholar
  13. 13.
    Heidig, S., Clarebout, G.: Do pedagogical agents make a difference to student motivation and learning? Educ. Res. Rev. 6(1), 27–54 (2011)CrossRefGoogle Scholar
  14. 14.
    Breitenstein, C., Lancker, D.V., Daum, I.: The contribution of speech rate and pitch variation to the perception of vocal emotions in a German and an American sample. Cogn. Emot. 15(1), 57–79 (2001)CrossRefGoogle Scholar
  15. 15.
    Ngiam, J., Lee, C.C.Y., Charumilind, J., Chen, Z.: U.S. Patent No. 9,741,392. U.S. Patent and Trademark Office, Washington, DC (2017)Google Scholar
  16. 16.
    Mayer, R.E.: Multimedia Learning. Academic Press, New York (2001)CrossRefGoogle Scholar
  17. 17.
    Mayer, R.E., Moreno, R.: Nine ways to reduce cognitive load in multimedia learning. Educ. Psychol. 38(1), 43–52 (2003)CrossRefGoogle Scholar
  18. 18.
    Veletsianos, G.: The impact and implications of virtual character expressiveness on learning and agent–learner interactions. J. Comput. Assist. Learn. 25(4), 345–357 (2009)CrossRefGoogle Scholar
  19. 19.
    Menges, R., Kumar, C., Müller, D., Sengupta, K.: GazeTheWeb: a gaze-controlled web browser. In: Proceedings of the 14th Web for All Conference on the Future of Accessible Work, p. 25. ACM (2017)Google Scholar
  20. 20.
    Gudykunst, W.B., Ting-Toomey, S., Chua, E.: Culture and Interpersonal Communication. Sage Publications Inc., Thousand Oaks (1988)Google Scholar
  21. 21.
    Hollien, H., Shipp, T.: Speaking fundamental frequency and chronologic age in males. J. Speech Lang. Hear. Res. 15(1), 155–159 (1972)CrossRefGoogle Scholar
  22. 22.
    Hsiao, T.Y., Solomon, N.P., Luschei, E.S., Titze, I.R.: Modulation of fundamental frequency by laryngeal muscles during vibrato. J. Voice 8(3), 224–229 (1994)CrossRefGoogle Scholar
  23. 23.
    Mizuno, O., Nakajima, S.Y.: A new synthetic speech/sound control language. In: Fifth International Conference on Spoken Language Processing, pp. 2007–2010 (1998)Google Scholar
  24. 24.
    Frechette, C., Moreno, R.: The roles of animated pedagogical agents’ presence and nonverbal communication in multimedia learning environments. J. Media Psychol. 22(2), 61–72 (2010)CrossRefGoogle Scholar
  25. 25.
    What’s Your Speech Rate?. Accessed 29 Nov 2018
  26. 26.
    Mehrabian, A.: Silent Messages, 2nd edn. Wadsworth, Belmont (1981)Google Scholar
  27. 27.
    Richmond, V.P., Gorham, J.S., McCroskey, J.C.: The relationship between selected immediacy behaviors and cognitive learning. Ann. Int. Communication Assoc. 10(1), 574–590 (1987)CrossRefGoogle Scholar
  28. 28.
    Richmond, V.P., McCroskey, J.C., Johnson, A.D.: Development of the nonverbal immediacy scale (NIS): measures of self-and other-perceived nonverbal immediacy. Commun. Q. 51(4), 504–517 (2003)CrossRefGoogle Scholar
  29. 29.
    Servilha, E.A.M., Costa, A.T.F.D.: Knowledge about voice and the importance of voice as an educational resource in the perspective of university professors. Revista CEFAC 17(1), 13–26 (2015)CrossRefGoogle Scholar
  30. 30.
    Scott, M.D., Wheeless, L.R.: Communication apprehension, student attitudes, and levels of satisfaction. W. J. Speech Commun. 41, 188–198 (1975)CrossRefGoogle Scholar
  31. 31.
    Gorham, J.: The relationship between verbal teacher immediacy behaviors and student learning. Commun. Educ. 37(1), 40–53 (1988)CrossRefGoogle Scholar
  32. 32.
    Chesebro, J.L., McCroskey, J.C.: The relationship between students’ reports of learning and their actual recall of lecture material: A validity test. Commun. Educ. 49, 297–301 (2000)CrossRefGoogle Scholar
  33. 33.
    Bartneck, C., Croft, E., Kulic, D.: Measuring the anthropomorphism, animacy, likeability, perceived intelligence and perceived safety of robots. In: Metrics for HRI Workshop, Technical report, vol. 471, pp. 37–44 (2008)Google Scholar
  34. 34.
    Medawela, R.S.H.B., Ratnayake, D.R.D.L., Abeyasinghe, W.A.M.U.L., Jayasinghe, R.D., Marambe, K.N.: Effectiveness of “fill in the blanks” over multiple choice questions in assessing final year dental undergraduates. Educación Médica 19(2), 72–76 (2018)CrossRefGoogle Scholar
  35. 35.
    Kelley, D.H., Gorham, J.: Effects of immediacy on recall of information. Commun. Educ. 37(3), 198–207 (1988)CrossRefGoogle Scholar
  36. 36.
    Christophel, D.M.: The relationships among teacher immediacy behaviors, student motivation, and learning. Commun. Educ. 39(4), 323–340 (1990)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sofia Fountoukidou
    • 1
    Email author
  • Uwe Matzat
    • 1
  • Jaap Ham
    • 1
  • Cees Midden
    • 1
  1. 1.Eindhoven University of TechnologyEindhovenThe Netherlands

Personalised recommendations