Skip to main content
SpringerLink
Log in

Role of emotional tone and gender of computer-generated voices in multimedia lessons

  • Research Article
  • Published:
Educational technology research and development Aims and scope Submit manuscript

Abstract

How does the emotional tone (happy or sad) and gender (male or female) of the instructor’s computer-generated voice in a multimedia lesson affect learning processes and outcomes for female learners (Experiment 1) and male learners (Experiment 2)? In two experiments, college students viewed a 2.5 min narrated slideshow on the formation of lightning and then took a posttest. In a 2 (emotion) × 2 (gender) between-subjects design, the instructor's computer-generated voice was either happy female, sad female, happy male, or sad male, as produced by text-to-speech software. Female students (in Experiment 1) who received happy instructors gave higher ratings of instructor happiness, their own felt happiness, and how engaging and facilitative the instructor was, indicating that they could perceive, feel, and socially relate to the voice’s emotion, and these effects were stronger for the female voice than the male voice. Male students (in Experiment 2) showed the same pattern for perceived emotion, partially for felt emotion, and not for social connection with the instructor. In terms of learning outcomes, happy instructors produced better transfer test scores only for males and did not produce better retention test scores for males or females. These results provide partial support for the positivity principle, which posits that people learn better from happy instructors than from sad instructors and indicate that the instructor's gender plays a moderating role.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available in the figshare repository, https://doi.org/10.6084/m9.figshare.20012471.v1.

References

  • Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64(3), 431–441. https://doi.org/10.1037/0022-3514.64.3.431

    Article  Google Scholar 

  • Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice. Contemporary Educational Psychology, 30, 117–139. https://doi.org/10.1016/j.cedpsych.2004.07.001

    Article  Google Scholar 

  • Barrett, L. F. (2017). How emotions are made. Hough Mifflin Harcourt.

    Google Scholar 

  • Baylor, A., Ryu, J. & Shen, E. (2003). The effects of pedagogical agent voice and animation on learning, motivation and perceived persona. In D. Lassner & C. McNaught (Eds.), Proceedings of ED-MEDIA 2003--World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 452–458). Honolulu, Hawaii, USA: Association for the Advancement of Computing in Education (AACE). Retrieved June 1, 2022, from https://www.learntechlib.org/primary/p/13800/.

  • Bower, G. H. (1992). How might emotions affect learning. In S. Christianson (Ed.), The handbook of emotion and memory: Research and theory (pp. 3–31). Psychology Press.

    Google Scholar 

  • Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick. Harvard University Press.

    Book  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Wiley.

    Book  Google Scholar 

  • Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250. https://doi.org/10.1080/1358165042000283101

    Article  Google Scholar 

  • Craig, S. D., & Schroeder, N. L. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education, 114, 193–205. https://doi.org/10.1016/j.compedu.2017.07.003

    Article  Google Scholar 

  • Craig, S. D., & Schroeder, N. L. (2018). Text-to-speech software and learning: Investigating the relevancy of the voice effect. Journal of Educational Computing Research., 57, 1534–1548. https://doi.org/10.1177/2F0735633118802877

    Article  Google Scholar 

  • D’Mello, S. K. (2017). Emotional learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (pp. 115–127). Society for Learning Analytics Research.

    Chapter  Google Scholar 

  • D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. https://doi.org/10.1016/j.learninstruc.2011.10.001

    Article  Google Scholar 

  • Fiorella, L. (2022). The embodiment principle in multimedia learning. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 286–295). Cambridge University Press.

    Google Scholar 

  • Fiorella, L., & Mayer, R. E. (2022). Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 277–285). Cambridge University Press.

    Google Scholar 

  • Graesser, A. C., Chipman, P., King, B., McDaniel, B., & D’Mello, S. K. (2007). Emotions and learning with auto tutor. Frontiers in Artificial Intelligence and Applications, 158, 569–571.

    Google Scholar 

  • Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, 106724. https://doi.org/10.1016/j.chb.2021.106724

    Article  Google Scholar 

  • Lawson, A. P., & Mayer, R. E. (2021). The power of voice to convey emotion in multimedia instructional messages. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-021-00282-y

    Article  Google Scholar 

  • Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021a). Do learners recognize and relate to the emotions displayed by virtual instructors? International Journal of Artificial Intelligence in Education, 31, 134–153. https://doi.org/10.1007/s40593-021-00238-2

    Article  Google Scholar 

  • Lawson, A., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021b). The positivity principle: Do positive instructors improve learning from instruction video lectures? Educational Technology Research and Development, 69, 3101–3129. https://doi.org/10.1007/s11423-021-10057-w

    Article  Google Scholar 

  • Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021). Recognizing the emotional state of human and virtual instructors. Computers in Human Behavior, 114, 106554. https://doi.org/10.1016/j.chb.2020.106554

    Article  Google Scholar 

  • Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997, March). The persona effect: affective impact of animated pedagogical agents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 359–366).

  • Loderer, K., Pekrun, R., & Lester, J. C. (2020). Beyond cold technology: A systematic review and meta-analysis on emotions in technology-based learning environments. Learning and instruction, 70, 101162. https://doi.org/10.1016/j.learninstruc.2018.08.002

    Article  Google Scholar 

  • Mayer, R. E. (2021). Multimedia learning (3rd ed.). Cambridge University Press.

    Google Scholar 

  • Mayer, R. E. (2020). Searching for the role of emotions in e-learning. Learning and Instruction, 70, 101213. https://doi.org/10.1016/j.learninstruc.2019.05.010

    Article  Google Scholar 

  • Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93(2), 390–397. https://doi.org/10.1037/0022-0663.93.2.390

    Article  Google Scholar 

  • Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology: Applied, 18(3), 239–252. https://doi.org/10.1037/a0028616

    Article  Google Scholar 

  • Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), 187–198. https://doi.org/10.1037/0022-0663.93.1.187

    Article  Google Scholar 

  • Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100(2), 380–386. https://doi.org/10.1037/0022-0663.100.2.380

    Article  Google Scholar 

  • Mayer, R. E., & Moreno, R. (1998). A spit-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312–320. https://doi.org/10.1037/0022-0663.90.2.312

    Article  Google Scholar 

  • Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of speaker’s voice. Journal of Educational Psychology, 95(2), 419–425. https://doi.org/10.1037/0022-0663.95.2.419

    Article  Google Scholar 

  • McGaugh, J. L. (2018). Emotional arousal regulation of memory consolidation. Current Opinion in Behavioral Sciences, 19, 55–60. https://doi.org/10.1016/j.cobeha.2017.10.003

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358–368. https://doi.org/10.1037/0022-0663.91.2.358

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (2000). Engaging students in active learning: The case for personalized multimedia messages. Journal of Educational Psychology, 93, 724–733.

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(4), 156–163. https://doi.org/10.1037/0022-0663.92.4.724

    Article  Google Scholar 

  • Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309–326. https://doi.org/10.1007/s10648-007-9047-2

    Article  Google Scholar 

  • Nass, C., & Brave, S. (2005). Wired for speech: How voice activates and advances the human-computer relationship. MIT Press.

    Google Scholar 

  • Pekrun, R. (2016). In K. R. Wentzel & D. B. Miele (Eds.), Academic emotions. In Handbook of motivation at school (2nd ed., pp. 120–144). Routledge.

  • Pekrun, R. (2017). Achievement emotions. In A. J. Elliot, C. S. Dweck, & D. S. Yeager (Eds.), Handbook of competence and motivation: Theory and application (pp. 251–271). The Guilford Press.

    Google Scholar 

  • Pekrun, R., & Perry, R. P. (2014). Control-value theory of achievement emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 120–141). Taylor and Francis.

    Chapter  Google Scholar 

  • Plant, E. A., Hyde, J. S., Keltner, D., & Devine, P. G. (2000). The gender stereotyping of emotions. Psychology of Women Quarterly, 24(1), 81–92.

    Article  Google Scholar 

  • Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Y. Tettegah & M. P. McCreery (Eds.), Emotions, Technology, and Learning (pp. 131–161). Academic Press.

    Chapter  Google Scholar 

  • Reeves, B., & Nass, C. (1996). The media equation. Cambridge University Press.

    Google Scholar 

  • Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255. https://doi.org/10.1111/2Fj.1467-9280.2006.01693.x

    Article  Google Scholar 

  • Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714

    Article  Google Scholar 

  • Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145–172. https://doi.org/10.1037/0033-295X.110.1.145

    Article  Google Scholar 

  • Schroeder, N. L., & Adesope, O. O. (2014). A systematic review of pedagogical agents’ persona, motivation, and cognitive load implications for learners. Journal of Research on Technology in Education, 46, 229–251. https://doi.org/10.1080/15391523.2014.888265

    Article  Google Scholar 

  • Shields, S. A. (2013). Gender and emotion: What we think we know, what we need to know, and why it matters. Psychology of Women Quarterly, 37(4), 423–435.

    Article  Google Scholar 

  • Wang, F., Li, W., & Zhao, T. (2022). Multimedia learning with animated pedagogical agents. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 450–460). Cambridge University Press.

    Google Scholar 

Download references

Funding

This project was supported by Grant 1821833 and Grant 2201020 from the National Science Foundation.

Author information

Authors and Affiliations

  1. Department of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, 93106, USA

    Fangzheng Zhao & Richard E. Mayer

Authors
  1. Fangzheng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard E. Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FZ and REM: both contributed to developing the design of the study, interpreting the results, and writing the manuscript. FZ: took responsibility for creating the materials, running the participants, and tabulating the data.

Corresponding author

Correspondence to Fangzheng Zhao.

Ethics declarations

Competing interest

The authors declare that there were no known competing financial interests or personal relationships that might have affected the work reported in this paper.

Ethical approval

The research was approved by the Institutional Review Board (IRB) at the University of California, Santa Barbara, and followed the standards for research involving human subjects of the American Psychological Association.

Informed consent

Participants provided informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, F., Mayer, R.E. Role of emotional tone and gender of computer-generated voices in multimedia lessons. Education Tech Research Dev 71, 1449–1469 (2023). https://doi.org/10.1007/s11423-023-10228-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-023-10228-x

Keywords

Search

Navigation