Abstract
There has been much research on the effectiveness of animated pedagogical agents in an educational context, however there is little research about how the emotions they display contribute to a learner’s understanding of the lesson. The positivity principle suggests that learners should learn better from instructors with positive emotions compared to those with negative emotions. Additionally, the media equation theory (Reeves and Nass 1996) would suggest this principle should be true for animated instructors as well. In an experiment, students viewed a lesson on binomial probability taught by an animated instructor who was happy (positive/active), content (positive/passive), frustrated (negative/active), or bored (negative/passive). Learners were able to recognize positive from negative emotions, rated the positive instructors as better at facilitating learning, more credible, more humanlike, and more engaging. Additionally, learners who saw positive instructors indicated they tried to pay attention to the lesson and enjoyed the lesson more than those who saw negative instructors. However, learners who saw positive instructors did not perform better on a delayed test than those who saw negative instructors. This suggests that learners recognize and react to the emotions of the virtual instructors, but research is needed to determine how the emotions displayed by virtual instructors can promote better learning outcomes.
This is a preview of subscription content, access via your institution.
References
Baylor, A. L., & Ryu, J. (2003). The API (Agent Persona Instrument) for assessing pedagogical agent persona. Technology, Instruction, Cognition & Learning, 2, 291–314.
Calvo, R. A., D’Mello, S., Gratch, J. M., & Kappas, A. (Eds.). (2015). The Oxford handbook of affective computing. New York: Oxford University Press.
Cassell, J., Sullivan, J., Churchill, E., & Prevost, S. (Eds.)., (2000). Embodied conversational agents. Cambridge: MIT Press.
Derry, S. J., Sherin, M. G., & Sherin, B. L. (2014). Multimedia learning with video. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 785–812). Cambridge: Cambridge University Press.
D’Mello, S., Lehman, B., Sullins, J., Daigle, R., Combs, R., Vogt, K., Perkins, L., & Graesser, A. (2010). A time for emoting: When affectsensitivity is and isn’t effective at promoting deep learning. International Conference on Intelligent Tutoring Systems (pp. 245–254).
D’Mello, S. D., Lehman, B., & Graesser, A. (2011). A motivationally supportive affectsensitive autotutor. New Perspectives on Affect and Learning Technologies, 3, 113–126.
Johnson, W. L., & Lester, J. C. (2016). Facetoface interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education, 26(1), 25–36.
Johnson, W. L., Rickel, J. W., & Lester, J. C. (2000). Animated pedagogical agents: Facetoface interaction in interactive learning environments. International Journal of Artificial intelligence in education, 11(1), 47–78.
Lawson, A. P., Mayer, R. E., AdamoVillani, N., Benes, B., Lei, X., & Cheng, J. (2021). Recognizing the emotional state of human and virtual instructors. Computers in Human Behavior, 114. https://doi.org/10.1016/j.chb.2020.106554.
Loderer, K., Pekrun, R., & Lester, J. (2021). Beyond cold technology: A systematic review and metaanalysis on emotions in technologybased learning environments. Learning and Instruction, 70. https://doi.org/10.1016/j.learninstruc.2018.08.002.
Mayer, R. E. (2011). Applying the science of learning. Boston: Pearson.
Mayer, R. E. (2014a). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 43–71). Cambridge: Cambridge University Press.
Mayer, R. E. (2014b). Principles based on social cues in multimedia learning: Personalization, voice, embodiment, and image principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 345–368). Cambridge: Cambridge University Press.
Mayer, R. E. (2020a). Multimedia learning (3rd ed.). Cambridge: Cambridge University Press.
Mayer, R. E. (2020b). Searching for the role of emotions in elearning. Learning and Instruction, 70. https://doi.org/10.1016/j.learninstruc.2019.05.010.
Mayer, R. E., & Greeno, J. G. (1972). Structural differences between outcomes produced by different instructional methods. Journal of Educational Psychology, 63(2), 165–173.
Mayer, R. E., & Estrella, G. (2014). Benefits of emotional design in multimedia instruction. Learning and Instruction, 33, 12–18.
Mayer, R. E., Johnson, L., Shaw, E., & Sahiba, S. (2006). Constructing computerbased tutors that are socially sensitive: Politeness in educational software. International Journal of Human Computer Studies, 64, 36–42.
Mayer, R. E., Fiorella, L., & Stull, A. (2020). Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development, 68, 837–852. https://doi.org/10.1007/s11423020097496.
Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309–326.
Pekrun, R., & Perry, R. P. (2014). Controlvalue theory of achievement emotions. In R. Pekrun & L. LinnenbrinkGarcia (Eds.), International handbook of emotions in education (pp. 120–141). New York: Taylor and Francis.
Pekrun, R., & LinnenbrinkGarcia, L. (2012). Academic emotions and student engagement. In Handbook of research on student engagement (pp. 259–282). Boston: Springer.
Plass, J. L., & Kaplan, U. (2015). Emotional design in digital media for learning. In S. Y. Tettegah & M. P. McCreery (Eds.), Emotions, Technology, and Learning (pp. 131–161). Cambridge: Academic.
Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Y. Tettegah & M. Gartmeier (Eds.), Emotions, technology, design, and learning (p. 131–161). Elsevier Academic Press. https://doi.org/10.1016/B9780128018569.000074.
Plass, J. L., Heidig, S., Hayward, E. O., Homer, B. D., & Um, E. (2014). Emotional design in multimedia learning: Effects of shape and color on affect and learning. Learning and Instruction, 29, 128–140.
Plass, J. L., Homer, B. D., MacNamara, A., Ober, T., Rose, M., Pawar, S., Hovey, C. M., & Olsen, A. (2020). Emotional design for digital games for learning: The affective quality of expression, color, shape, and dimensionality. Learning and Instruction, 70. https://doi.org/10.1016/j.learninstruc.2019.01.005.
Reeves, B., & Nass, C. (1996). The media equation. New York: Cambridge University Press.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178.
Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172.
Um, E. R., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104(2), 485–498.
Wang, N., Johnson, W. L., Mayer, R. E., Rizzo, P., Shaw, E., & Collins, H. (2008). The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human Computer Studies, 66, 96–112.
Woolf, B. P., Arroyo, I., Cooper, D., Burleson, W., & Muldner, K. (2010). Affective tutors: Automatic detection of the response to student emotion. In R. Nkambou, R. Mizoguchi, & J. Bourdeau (Eds.), Advances in Intelligent Tutoring Systems (pp. 207–227). Berlin: Springer.
Acknowledgements
This project was supported by Grant 1,821,833 from the National Science Foundation.
Author information
Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors report no conflicts of interest.
Appendix A Video Script
Appendix A Video Script
Hi everyone. Imagine that you are trying to impress your friends with your ability to predict what will happen if you roll a die a certain number of times. For example, suppose you win if you roll 5 or 6 and you lose if you roll 1, 2, 3, or 4. Let’s say you roll the die 5 times and you win 2 times and lose 3 times. What exactly is the probability of that happening? Today, I will help you understand how to answer questions like this one. This is called binomial probability.
First, you need to understand trials and outcomes. A trial is something you do. For example, you roll a die. An outcome is what happens on the trial. For instance, if you roll a die (the trial), the outcome could be that you rolled a 4.
Second, you also need to think about success and failure. A success is defined, by you, as one or more of the possible outcomes. For example, a success of rolling the die could be that you roll a number greater than 4. That means, if you roll a die, and get a 5 or 6, a success has occurred. On the other hand, a failure occurs on any trial that is not a success. So, if you defined success as rolling a number greater than 4, failure would occur if you rolled a 1, 2, 3, or 4.
Next, we should figure out the probability of success. The probability of success is the number of success outcomes divided by the total number of outcomes (including the success outcomes) if all the outcomes have an equal chance. In this case, there are 6 equally likely outcomes and 2 of them are successes, so the probability of success is 2 out of 6 or onethird. We can expect a 5 or a 6 to come up on about onethird of the times the die is rolled. The probability of success can be symbolized by the letter P.
Similarly, there is a probability of failure. This is the probability of success subtracted from 1. So, in our example, the probability of failure is 1 minus onethird which is twothirds. The probability of failure can be symbolized as 1 minus P.
Now you know how to determine the probability of success (symbolized as P) and the probability of failure (symbolized as 1 minus P).
The next concept you need to know is sequence. A sequence is what happens when you conduct several trials, one after another, like rolling a die 5 times in a row. For each trial, we have either a success or a failure, so the sequence reports what occurred. For example, say we rolled a die 5 times in a row and rolled a 2, then a 4, then a 6, then a 2, and then a 5. The sequence would be failure, failure, success, failure, success.
A sequence, like the previous example, has a probability of occurring, which is called the joint probability of a sequence. This can be found by multiplying the probabilities of each individual event. Let’s take the previous example. We had failure, failure, success, failure, success. Now, we multiply the probability of each happening, so we get twothirds (for failure), times twothirds (for failure), times onethird (for success), times twothirds (for failure), times onethird (for success). We can also write this as onethird squared times twothirds cubed. So, the joint probability of this particular sequence occurring is 8 out of 243.
We can compute the probability for any specific sequence. So, let’s say the number of trials in a sequence can be symbolized by the letter N and the number of successes in those trials is called R and the number of failures is N minus R. To figure out the probability of any sequence, you can use the formula displayed on the screen. We multiply the probability of success (P) by itself R times, then we multiply the probability of failure (1 minus P) by itself N minus R times, and we finally multiply those two numbers together. This is called the joint probability of a sequence.
Now you know how to compute the joint probability of a sequence of successes and failures. The next step is to figure out how many different sequences (that is, patterns of successes and failures) have that same number of successes out of N trials. For example, there are three different ways that we can have 2 successes from 3 trials:

success, success, failure.

success, failure, success.

failure, success, success.
As you can see, in each sequence, there are 2 successes and 1 failure. The number of different sequences having R successes in N trials is called the number of combinations. In this example, there are 3 combinations for a sequence having 2 successes out of 3 trials.
The number of combinations may be simple to work out by hand when there are just a few trials, like our previous example, but what if I asked you how many different combinations can occur for 2 successes in 5 trials? In cases like this, having a formula to find the number of combinations is quite helpful. This formula is N factorial divided by R factorial times N minus R factorial. This equation includes a factorial symbol (indicated by an exclamation point). This factorial symbol means multiply the number before the exclamation mark times the number minus one, then times the number minus two, and so on down to 1. For example, 5 factorial equals 5 times 4 times 3 times 2 times 1, which equals 120.
Now, let’s finish finding the number of combinations that can occur for 2 successes in 5 trials. So, 5 factorial is equal to 120, which we just found out. Then, we divided that by 2 factorial (which is 2 times 1) times 5 − 2 factorial, or 3 factorial (which is 3 times 2 times 1). That gives us 120 divided by 12, which equals 10. This means there are 10 ways to get 2 successes in 5 trials.
Now you see how to compute the joint probability of a particular sequence that has R successes in N trials (such as failure, failure, success, failure, success) how to compute the number of combinations in which a sequence has R successes in N trials (such as 10 ways to get 2 successes out of 5 trials).
As the final step in computing binomial probability you just put those two parts together. You can figure out the probability of getting R successes out of N trials by multiplying the number of combinations for a sequence that has R successes out of N trials by the joint probability of any one of those sequences. When you do this, you are finding the probability of R successes in N trials. So, if you put that all together you get the formula on the screen. This is what we call a binomial probability.
Let’s do an example to see how well our formula for binomial probability works. Suppose I want to find the probability of rolling a die 5 times and having 5 or 6 come up exactly 2 times. In this case, we want to know the binomial probability of 2 successes in 5 trials, when the probability of success is 1/3. The binomial probability equals the number of combinations that have 2 successes in 5 trials times the joint probability of this sequence.
The number of combinations is 5 factorial divided by 2 factorial times 3 factorial, which equals 5 times 4 times 3 times 2 times 1 divided by 2 times 1 times 3 times 2 times 1 which equals 120 divided by 12 which is 10.
The joint probability of any one sequence is onethird times onethird times twothirds times twothirds times twothirds, which equals 8 divided by 243.
Multiply the number of combinations times the joint probability of a sequence. We get 10 times 8 divided by 243 which equals 80 divided by 243 (or about 0.33).
This means you have about a onethird chance of rolling a die 5 times and getting 5 or 6 to come up exactly 2 times.
Now you know how to determine the probability of R successes out of N trials when the probability of success is P.
Rights and permissions
About this article
Cite this article
Lawson, A.P., Mayer, R.E., AdamoVillani, N. et al. Do Learners Recognize and Relate to the Emotions Displayed By Virtual Instructors?. Int J Artif Intell Educ (2021). https://doi.org/10.1007/s40593021002382
Received:
Revised:
Accepted:
Published:
Keywords
 Animated pedagogical agents
 Affective processes
 Emotion
 Emotional tone
 Learning