Abstract
Computer-based simulations for learning offer affordances for advanced capabilities and expansive possibilities for knowledge construction and skills application. Virtual agents, when powered by artificial intelligence (AI), can be used to scaffold personalized and adaptive learning processes. However, a synthesis or a systematic evaluation of the learning effectiveness of AI-powered virtual agents in computer-based simulations for learning is still lacking. Therefore, this meta-analysis is aimed at evaluating the effects of AI-powered virtual agents in computer-based simulations for learning. The analysis of 49 effect sizes derived from 22 empirical studies suggested a medium positive overall effect, \(\overline{g }=0.43\), SE = 0.08, 95% C.I. [0.27, 0.59], favoring the use of AI-powered virtual agents over the non-AI-powered virtual agent condition in computer-based simulations for learning. Further, moderator analyses revealed that intervention length, AI technologies, and the representation of AI-powered virtual agents significantly explain the heterogeneity of the overall effects. Conversely, other moderators, including education level, domain, the role of AI-powered virtual agents, the modality of AI-powered virtual agents, and learning environment, appeared to be universally effective among the studies of AI-powered virtual agents in computer-based simulations for learning. Overall, this meta-analysis provides systematic and existing evidence supporting the adoption of AI-powered virtual agents in computer-based simulations for learning. The findings also inform about evidence-based design decisions on the moderators analyzed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Studies included in the meta-analysis are denoted by an asterisk mark in the references.
Al-Elq, A. H. (2010). Simulation-based medical teaching and learning. Journal of family and Community Medicine, 17(1), 35–40. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3195067/pdf/JFCM-17-35.pdf
Alinier, G. (2007). A typology of educationally focused medical simulation tools. Medical Teacher, 29(8), e243–e250. https://doi.org/10.1080/01421590701551185
Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation (Vol. 2, pp. 89–195). Academic Press. https://doi.org/10.1016/S0079-7421(08)60422-3
Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450–457. https://doi.org/10.1016/j.chb.2008.10.008
*Beidel, D. C., Tuerk, P. W., Spitalnick, J., Bowers, C. A., & Morrison, K. (2021). Treating childhood social anxiety disorder with virtual environments and serious games: a randomized trial. Behavior Therapy, 52(6), 1351-1363. https://doi.org/10.1016/j.beth.2021.03.003
Bjork, E. L., & Bjork, R. A. (2011). Making things hard on your-self, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R. W. Pew, L. M. Hough, & J. R. Pomerantz (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 56–64). Worth Publishers.
Byrne, J., Heavey, C., & Byrne, P. J. (2010). A review of Web-based simulation and supporting tools. Simulation Modelling Practice and Theory, 18(3), 253–276. https://doi.org/10.1016/j.simpat.2009.09.013
*Carlotto, T., & Jaques, P. A. (2016). The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment. International Journal of Human-Computer Studies, 95, 15-26. https://doi.org/10.1016/j.ijhcs.2016.06.001
Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(8), 723–733. https://doi.org/10.1177/016146816306400801
Cassell, J. (2001). Embodied conversational agents: Representation and intelligence in user interfaces. AI Magazine, 22(4), 67–83. https://doi.org/10.1609/aimag.v22i4.1593
Castro-Alonso, J. C., & Sweller, J. (2022). The modality principle in multimedia learning. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 261–267). Cambridge University Press. https://doi.org/10.1017/9781108894333.026
Castro-Alonso, J. C., Wong, R. M., Adesope, O. O., & Paas, F. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review, 33(3), 989–1015. https://doi.org/10.1007/s10648-020-09587-1
*Chen, C.-H., & Chou, M.-H. (2015). Enhancing middle school students' scientific learning and motivation through agent‐based learning. Journal of Computer Assisted Learning, 31(5), 481-492. https://doi.org/10.1111/jcal.12094
Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, 100002. https://doi.org/10.1016/j.caeai.2020.100002
*Cheng, Y. M., Chen, L. S., Huang, H. C., Weng, S. F., Chen, Y. G., & Lin, C. H. (2009). Building a general purpose pedagogical agent in a web-based multimedia clinical simulation system for medical education. IEEE Transactions on Learning Technologies, 2(3), 216-225. https://doi.org/10.1109/TLT.2009.18
Chernikova, O., Heitzmann, N., Stadler, M., Holzberger, D., Seidel, T., & Fischer, F. (2020). Simulation-based learning in higher education: A meta-analysis. Review of Educational Research, 90(4), 499–541. https://doi.org/10.3102/0034654320933544
*Clarebout, G., & Elen, J. (2006). Open learning environments and the impact of a pedagogical agent. Journal of Educational Computing Research, 35(3), 211-226. https://doi.org/10.2190/3UL1-4756-H837-2704
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
* D'Mello, S., Olney, A., Williams, C., & Hays, P. (2012). Gaze tutor: A gaze-reactive intelligent tutoring system. International Journal of Human-computer studies, 70(5), 377-398. https://doi.org/10.1016/j.ijhcs.2012.01.004
Dai, C.-P. (2023). Enhancing learning achievements and self-efficacy for preservice teachers using model-based support in simulation-based learning with artificial intelligence-powered virtual agents. Doctoral dissertation. Florida State University.
Dai, C.-P. (2024). Applying machine learning to augment the design and assessment of immersive learning experience. In M. S. Khine (Ed.), Machine learning in educational sciences: Approaches, applications and advances. Springer. https://doi.org/10.1007/978-981-99-9379-6_12
Dai, C.-P., & Ke, F. (2022). Educational applications of artificial intelligence in simulation-based learning: A systematic mapping review. Computers & Education: Artificial Intelligence, 3, 100087. https://doi.org/10.1016/j.caeai.2022.100087
Dai, Z., Ke, F., Dai, C.-P., Pachman, M., & Yuan, X. (2021). Role-play in virtual reality: A teaching training design case using OpenSimulator. In G. Akcayir & C. D. Epp (Eds.), Designing, deploying, and evaluating virtual and augmented reality in education (pp. 143–163). IGI Global. https://doi.org/10.4018/978-1-7998-5043-4.ch007
Dai, C.-P., Ke, F., Dai, Z., & Pachman, M. (2023). Improving teaching practices via virtual reality-supported simulation-based learning: Scenario design and the duration of implementation. British Journal of Educational Technology, 54(4), 836–856. https://doi.org/10.1111/bjet.13296
Dai, C-P., Ke, F., Zhang, N., Barrett, A., West, L., Bhowmik, S., Southerland, S. A., & Yuan, X. (2024). Designing conversational agents to support student teacher learning in virtual reality simulation: A case study. In Proceedings of The ACM CHI conference on Human Factors in Computing Systems Extended Abstracts (ACM CHI EA ‘24), Honolulu. https://doi.org/10.1145/3613905.3637145
Daradoumis, T., & Arguedas, M. (2020). Cultivating students’ reflective learning in metacognitive activities through an affective pedagogical agent. Educational Technology & Society, 23(2), 19–31. https://www.jstor.org/stable/26921131
Davis, R. O. (2018). The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis. Educational Research Review, 24, 193–209. https://doi.org/10.1016/j.edurev.2018.05.002
de Jong, T., & Van Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179–201. https://doi.org/10.3102/00346543068002179
Decker, S., Sportsman, S., Puetz, I., & Billings, I. (2008). The evolution of simulation and its contribution to competency. The Journal of Continuing Education in Nursing, 39(2), 74–80. https://doi.org/10.3928/00220124-20080201-06
Deeks, J. J., Higgins, J. P., Altman, D. G., Cochrane Statistical Methods Group. (2019). Analysing data and undertaking meta-analyses. In J. P. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (2nd ed., pp. 241–284). John Wiley & Sons. https://doi.org/10.1002/9781119536604.ch10
*Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74-100. https://doi.org/10.1016/j.compedu.2017.04.005
Domagk, S. (2010). Do pedagogical agents facilitate learner motivation and learning outcomes?: The role of the appeal of agent’s appearance and voice. Journal of Media Psychology: Theories, Methods, and Applications, 22(2), 84–97. https://doi.org/10.1027/1864-1105/a000011
Duval, S., & Tweedie, R. (2000). A nonparametric ‘trim and fill’ method of assessing publication bias in meta-analysis. Journal of the American Statistical Association, 95(449), 89–98. https://doi.org/10.1080/01621459.2000.10473905
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867–872. https://doi.org/10.1016/0140-6736(91)90201-Y
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629
Elkot, M. A. (2019). Embedding adaptation levels within intelligent tutoring systems for developing programming skills and improving learning efficiency. International Journal of Advanced Computer Science and Applications, 10(12), 82–87. https://doi.org/10.14569/IJACSA.2019.0101211
*Frechette, M. C. (2008). Animated pedagogical agents: How the presence and nonverbal communication of a virtual instructor affect perceptions and learning outcomes in a computer-based environment about basic physics concepts [Unpublished doctoral dissertation]. The University of New Mexico.
Gamer, M., Lemon, J., & Fellows, I. (2012). irr: Various coefficients of interrater reliability and agreement. R package version 0.84.1. Retrieved from https://cran.r-project.org/web/packages/irr/index.html
Gibbons, A. S. (2020). What is instructional strategy? Seeking hidden dimensions. Educational Technology Research and Development, 68(6), 2799–2815. https://doi.org/10.1007/s11423-020-09820-2
*Goldberg, B., & Cannon-Bowers, J. (2015). Feedback source modality effects on training outcomes in a serious game: Pedagogical agents make a difference. Computers in Human Behavior, 52, 1-11. https://doi.org/10.1016/j.chb.2015.05.008
Gorbunova, A., van Merrienboer, J.J.G. & Costley, J. (2023). Are inductive teaching methods compatible with cognitive load theory?. Educational Psychology Review, 35, Article 111. https://doi.org/10.1007/s10648-023-09828-z
Graesser, A. C. (2016). Conversations with AutoTutor help students learn. International Journal of Artificial Intelligence in Education, 26(1), 124–132. https://doi.org/10.1007/s40593-015-0086-4
Graesser, A. C., Jeon, M., & Dufty, D. (2008). Agent technologies designed to facilitate interactive knowledge construction. Discourse Processes, 45(4–5), 298–322. https://doi.org/10.1080/01638530802145395
*Gulz, A., Haake, M., & Silvervarg, A. (2011). Extending a teachable agent with a social conversation module–effects on student experiences and learning. In International conference on artificial intelligence in education (pp. 106–114). Springer, Berlin, Heidelberg. https://dl.acm.org/doi/10.5555/2026506.2026524
Guo, Y. R., & Goh, D. H. L. (2015). Affect in embodied pedagogical agents: Meta-analytic review. Journal of Educational Computing Research, 53(1), 124–149. https://doi.org/10.1177/0735633115588774
Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107
Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486–504.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5
Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6(1), 27–54. https://doi.org/10.1016/j.edurev.2010.07.004
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
Higgins, J. P., Li, T., & Deeks, J. J. (2019). Choosing effect measures and computing estimates of effect. In J. P. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (2nd ed., pp. 143–176). John Wiley & Sons. https://doi.org/10.1002/9781119536604.ch6
Hwang, G. J., Xie, H., Wah, B. W., & Gašević. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001
Jackson, G. T., & McNamara, D. S. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105(4), 1036–1049. https://doi.org/10.1037/a0032580
Johnson, W. L. (2003). Interaction tactics for socially intelligent pedagogical agents. In Proceedings of the 8th International Conference on Intelligent User Interfaces (pp. 251–253). https://doi.org/10.1145/604045.604090
*Johnson, E. (2021). An intelligent tutoring system’s approach for negotiation training. [Unpublished doctoral dissertation]. University of Southern California.
Johnson, W. L., & Lester, J. C. (2016). Face-to-face interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education, 26(1), 25–36. https://doi.org/10.1007/s40593-015-0065-9
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415
Kane, M. J., & Engle, R. W. (2000). Working-memory capacity, proactive interference, and divided attention: Limits on long-term memory retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(2), 336–358. https://doi.org/10.1037/0278-7393.26.2.336
Ke, F., Dai, Z., Dai, C.-P., Pachman, M., Chaulagain, R., & Yuan, X. (2020). Designing virtual agents for simulation-based learning in virtual reality. In R. Zheng (Ed.), Cognitive and affective perspectives on immersive technology in education (pp. 151–170). IGI Global. https://doi.org/10.4018/978-1-7998-3250-8.ch008
Kim, S. (2005). The effect of multimedia design elements on learning outcomes in pedagogical agent research: a meta-analysis. In M. Grandbastien (Ed.), Young Research Track Proceedings of 12th International Conference on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology (pp. 69–76).
Kim, Y., & Baylor, A. L. (2016). Research-based design of pedagogical agent roles: A review, progress, and recommendations. International Journal of Artificial Intelligence in Education, 26(1), 160–169. https://doi.org/10.1007/s40593-015-0055-y
Kolb, D. A. (1984). Experiential learning experience as a source of learning and development. Prentice Hall.
Kolodner, J. L. (1992). An introduction to case-based reasoning. Artificial Intelligence Review, 6(1), 3–34. https://doi.org/10.1007/BF00155578
*Kron, F. W., Fetters, M. D., Scerbo, M. W., White, C. B., Lypson, M. L., Padilla, M. A., ... & Becker, D. M. (2017). Using a computer simulation for teaching communication skills: A blinded multisite mixed methods randomized controlled trial. Patient education and counseling, 100(4), 748-759. https://doi.org/10.1016/j.pec.2016.10.024
Kulik, J. A., & Fletcher, J. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi.org/10.3102/0034654315581420
Lane, H. C., Cahill, C., Foutz, S., Auerbach, D., Noren, D., Lussenhop, C., & Swartout, W. (2013). The effects of a pedagogical agent for informal science education on learner behaviors and self-efficacy. In Proceedings of International conference on artificial intelligence in education (pp. 309–318). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39112-5_32
*Le, N. T., & Wartschinski, L. (2018). A cognitive assistant for improving human reasoning skills. International Journal of Human-Computer Studies, 117, 45-54. https://doi.org/10.1016/j.ijhcs.2018.02.005
*Lin, L., Atkinson, R. K., Christopherson, R. M., Joseph, S. S., & Harrison, C. J. (2013). Animated agents and learning: Does the type of verbal feedback they provide matter? Computers & Education, 67, 239–249. https://doi.org/10.1016/j.compedu.2013.04.017
Lippert, A., Shubeck, K., Morgan, B., Hampton, A., & Graesser, A. (2020). Multiple agent designs in conversational intelligent tutoring systems. Technology, Knowledge and Learning, 25, 443–463. https://doi.org/10.1007/s10758-019-09431-8
Luck, M., & Aylett, R. (2000). Applying artificial intelligence to virtual reality: Intelligent virtual environments. Applied Artificial Intelligence, 14(1), 3–32. https://doi.org/10.1080/088395100117142
Maroengsit, W., Piyakulpinyo, T., Phonyiam, K., Pongnumkul, S., Chaovalit, P., & Theeramunkong, T. (2019). A survey on evaluation methods for chatbots. In Proceedings of the 2019 7th International Conference on Information and Education Technology (pp. 111–119). https://doi.org/10.1145/3323771.3323824
Mascarenhas, S., Guimarães, M., Prada, R., Dias, J., Santos, P. A., Star, K., Hirsh, B., Spice, E., & Kommeren, R. (2018). A virtual agent toolkit for serious games developers. In 2018 IEEE Conference on Computational Intelligence and Games (CIG) (pp. 1–7). IEEE. https://doi.org/10.1109/CIG.2018.8490399
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery learning? American Psychologist, 59(1), 14–19. https://doi.org/10.1037/0003-066X.59.1.14
McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI Magazine, 27(4), 12–14. https://doi.org/10.1609/aimag.v27i4.1904
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40. https://doi.org/10.1016/j.compedu.2013.07.033
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97. https://doi.org/10.1037/h0043158
*Moundridou, M., & Virvou, M. (2002). Evaluating the persona effect of an interface agent in a tutoring system. Journal of Computer Assisted Learning, 18(3), 253–261. https://doi.org/10.1046/j.0266-4909.2001.00237.x
Noetel, M., Griffith, S., Delaney, O., Harris, N. R., Sanders, T., Parker, P., del Pozo Cruz, B., & Lonsdale, C. (2022). Multimedia design for learning: An overview of reviews with meta-meta-analysis. Review of Educational Research, 92(3), 413–454. https://doi.org/10.3102/00346543211052329
Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427–469. https://doi.org/10.1007/s40593-014-0029-5
Pavlik Jr., P. I., Brawner, K., Olney, A., & Mitrovic, A. (2013). A review of student models used in intelligent tutoring systems. In R. A. Sottilare, A. Graesser, X. Hu, & H. Holden (Eds.), Design recommendations for intelligent tutoring systems: Learner modeling (Vol. 1, pp. 39–67). U.S. Army Research Laboratory. Retrieved from: https://gifttutoring.org/documents/42
Peng, T. H., & Wang, T. H. (2022). Developing an analysis framework for studies on pedagogical agent in an e-Learning environment. Journal of Educational Computing Research, 6(3). https://doi.org/10.1177/07356331211041701
Peterson, M. (2010). Computerized games and simulations in computer-assisted language learning: A meta-analysis of research. Simulation & Gaming, 41(1), 72–93. https://doi.org/10.1177/1046878109355684
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.R-project.org/
Reinwein, J. (2012). Does the modality effect exist? And if so, which modality effect? Journal of Psycholinguistic Research, 41(1), 1–32. https://doi.org/10.1007/s10936-011-9180-4
Richey, R. C., Klein, J. D., & Tracey, M. W. (2011). The instructional design knowledge base: Theory, research, and practice. Routledge.
Rickel, J. (2001). Intelligent virtual agents for education and training: Opportunities and challenges. In de Antonio, A., Aylett, R., Ballin, D. (Eds). Intelligent Virtual Agents. IVA 2001. Lecture Notes in Computer Science, vol 2190 (pp. 15–22). Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44812-8_2
Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26(2), 582–599. https://doi.org/10.1007/s40593-016-0110-3
Rosé, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. https://doi.org/10.1111/bjet.12858
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological bulletin, 86(3).
*Rosenthal-von der Pütten, A. M., Straßmann, C., & Krämer, N. C. (2016, September). Robots or agents–neither helps you more or less during second language acquisition. In Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (Eds). Proceedings of International conference on intelligent virtual agents (pp. 256–268). Springer, Cham. https://doi.org/10.1007/978-3-319-47665-0_23
Roth, P. L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47(3), 537–560. https://doi.org/10.1111/j.1744-6570.1994.tb01736.x
Russell, S. J., & Norvig, P. (Ed.). (2021). Artificial intelligence: A modern approach (4th edition). Pearson.
Scheel, A. M., Schijen, M. R., & Lakens, D. (2021). An excess of positive results: Comparing the standard psychology literature with registered reports. Advances in Methods and Practices in Psychological Science, 4(2), 1–12. https://doi.org/10.1177/25152459211007467
Schroeder, N. L., & Gotch, C. M. (2015). Persisting issues in pedagogical agent research. Journal of Educational Computing Research, 53(2), 183–204. https://doi.org/10.1177/0735633115597625
Schroeder, N. L., Adesope, O. O., & Gilbert, R. B. (2013). How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research, 49(1), 1–39. https://doi.org/10.2190/EC.49.1.a
Schweppe, J., & Rummer, R. (2014). Attention, working memory, and long-term memory in multimedia learning: An integrated perspective based on process models of working memory. Educational Psychology Review, 26(2), 285–306. https://doi.org/10.1007/s10648-013-9242-2
Seshia, S. A., Sadigh, D., & Sastry, S. S. (2022). Toward verified artificial intelligence. Communications of the ACM, 65(7), 46–55. https://doi.org/10.1145/3503914
*Shiban, Y., Schelhorn, I., Jobst, V., Hörnlein, A., Puppe, F., Pauli, P., & Mühlberger, A. (2015). The appearance effect: Influences of virtual agent features on performance and motivation. Computers in Human Behavior, 49, 5–11. https://doi.org/10.1016/j.chb.2015.01.077
Sinatra, A. M., Pollard, K. A., Files, B. T., Oiknine, A. H., Ericson, M., & Khooshabeh, P. (2021). Social fidelity in virtual agents: Impacts on presence and learning. Computers in Human Behavior, 114, 106562. https://doi.org/10.1016/j.chb.2020.106562
Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64(2), 489–528. https://doi.org/10.1111/j.1744-6570.2011.01190.x\
Spronck, P., Ponsen, M., Sprinkhuizen-Kuyper, I., & Postma, E. (2006). Adaptive game AI with dynamic scripting. Machine Learning, 63(3), 217–248. https://doi.org/10.1007/s10994-006-6205-6
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. https://doi.org/10.1016/0959-4752(94)90003-5
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
*Tanana, M. J., Soma, C. S., Srikumar, V., Atkins, D. C., & Imel, Z. E. (2019). Development and evaluation of ClientBot: Patient-like conversational agent to train basic counseling skills. Journal of Medical Internet Research, 21(7), e12529. https://doi.org/10.2196/12529
*Thompson, N., & McGill, T. J. (2017). Genetics with Jean: The design, development and evaluation of an affective tutoring system. Educational Technology Research and Development, 65(2), 279–299. https://doi.org/10.1007/s11423-016-9470-5
Tokac, U., Novak, E., & Thompson, C. G. (2019). Effects of game-based learning on students’ mathematics achievement: A meta-analysis. Journal of Computer Assisted Learning, 35(3), 407–420. https://doi.org/10.1111/jcal.12347
Tun, J. K., Alinier, G., Tang, J., & Kneebone, R. L. (2015). Redefining simulation fidelity for healthcare education. Simulation & Gaming, 46(2), 159–174. https://doi.org/10.1177/1046878115576103
*van der Meij, H. (2013). Do pedagogical agents enhance software training?. Human–Computer Interaction, 28(6), 518–547. https://doi.org/10.1080/07370024.2013.789348
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369
Veletsianos, G., & Russell, G. S. (2013). What do learners and pedagogical agents discuss when given opportunities for open-ended dialogue? Journal of Educational Computing Research, 48(3), 381–401. https://doi.org/10.2190/EC.48.3.e
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of statistical software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03
Vygotsky, L.S. (1978). Mind in society: Development of higher psychological processes. Harvard university press.
Wang, N., Johnson, W. L., Mayer, R. E., Rizzo, P., Shaw, E., & Collins, H. (2008). The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human-Computer Studies, 66(2), 98–112. https://doi.org/10.1016/j.ijhcs.2007.09.003
*Wang, F., Li, W., Mayer, R. E., & Liu, H. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110(2), 250-268. https://doi.org/10.1037/edu0000221
*Ward, W., Cole, R., Bolaños, D., Buchenroth-Martin, C., Svirsky, E., & Weston, T. (2013). My science tutor: A conversational multimedia virtual tutor. Journal of Educational Psychology, 105(4), 1115–1125. https://doi.org/10.1037/a0031589
Williamson, B., & Eynon, R. (2020). Historical threads, missing links, and future directions in AI in education. Learning, Media and Technology, 45(3), 223–235. https://doi.org/10.1080/17439884.2020.1798995
Wood, D., Bruner, J., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17, 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
Woolf, B. P., Lane, H. C., Chaudhri, V. K., & Kolodner, J. L. (2013). AI grand challenges for education. AI Magazine, 34(4), 66–84. https://doi.org/10.1609/aimag.v34i4.2490
Acknowledgements
The work is most closely related to the National Science Foundation grant 2110777. Any opinions, findings, and conclusions or recommendations expressed in these materials are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dai, CP., Ke, F., Pan, Y. et al. Effects of Artificial Intelligence-Powered Virtual Agents on Learning Outcomes in Computer-Based Simulations: A Meta-Analysis. Educ Psychol Rev 36, 31 (2024). https://doi.org/10.1007/s10648-024-09855-4
Accepted:
Published:
DOI: https://doi.org/10.1007/s10648-024-09855-4