Skip to main content
SpringerLink
Log in

The Impact of Game-Like Features on Learning from an Intelligent Tutoring System

  • Original research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

Prior research has shown that students learn from Intelligent Tutoring Systems (ITS). However, students’ attention may drift or become disengaged with the task over extended amounts of instruction. To remedy this problem, researchers have examined the impact of game-like features (e.g., a narrative) in digital learning environments on motivation and learning. Some of this research has concluded that the game-like features decrease learning because the features take away resources from the primary task of learning subject-matter content. However, these experiments have involved short-term interventions of less than an hour. Two experiments using college students examined the impact of adding game-like features to the ITS AutoTutor in an intervention that lasted 4 h. In one study, a game-like version was compared to a text-only version and a “do nothing” control. In another study, a game-like version was compared to a nongame version that had similar interfaces. Unlike prior research that has shown that narratives decrease learning in digitally-based learning environments, the game-like features, which included a narrative, had little impact on learning from the ITS. Reasons for the discrepancies are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Adams, D. M., & Clark, D. B. (2014). Integrating self-explanation functionality into a complex game environment: Keeping gaming in motion. Computers & Education, 73, 149–159.

    Article  Google Scholar 

  • Adams, D. M., Mayer, R. E., MacNamara, A., Koenig, A., & Wainess, R. (2012). Narrative games for learning: Testing the discovery and narrative hypotheses. Journal of Educational Psychology, 104, 235–249.

    Article  Google Scholar 

  • Baker, R. S. J., Corbett, A. T., Koedinger, K. R., Evenson, S., Roll, Il, Wagner, A. Z., et al. (2006). Adapting to when students game an Intelligent Tutoring System. In M. Ikeda, K. D. Ashley, & T. W. Chan (Eds.), Proceedings of Intelligent Tutoring Systems 8th international conference ITS 2011, May 2011 (pp. 392–401). Berlin: Springer.

  • Baker, R. S. J. D., D’Mello, S. K., Rodrigo, M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human–Computer Studies, 68, 223–241.

    Article  Google Scholar 

  • Biswas, G., Jeong, H., Kinnebrew, J., Sulcer, B., & Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology-Enhanced Learning, 5, 123–152.

    Article  Google Scholar 

  • Blizzard Entertainment. (2004). World of warcraft. http://www.google.com/?gws_rd=ssl#q=world+of+warcraft&stick=H4sIAAAAAAAAAONgFuLQz9U3MMyqsFTiBLGMDFPKKrREwjJTUvPdE3NTfROzU4tCwEwAMaI3Ni0AAAA.

  • ChanMin, K., & Pekrun, R. (2013). Emotions and motivation in learning and performance. Handbook of research on educational communications and technology (pp. 65–75). New York: Springer.

    Google Scholar 

  • D’Mello, S., & Graesser, A. C. (2012). Emotions during learning with AutoTutor. In P. J. Durlach & A. Lesgold (Eds.), Adaptive technologies for training and education. Cambridge: Cambridge University Press.

    Google Scholar 

  • Dzikovska, M. O., Steinhauser, N., Farrow, E., Moore, J. D., & Campbell, G. E. (2014). BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics. International Journal of Artificial Intelligence in Education, 24, 284–332.

    Article  Google Scholar 

  • Forsyth. (2014). Predicting learning: A fine-grained analysis of learning in a serious game. Unpublished doctoral dissertation. The University of Memphis.

  • Gee, J. P. (2009). Deep learning properties of good video games: How far can they go? In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 67–82). New York: Routledge.

    Google Scholar 

  • Gee, J. P. (2013). Games for learning. Educational Horizons, 91, 17–20.

    Google Scholar 

  • Graesser, A. C. (2016). Conversations with AutoTutor help students learn. International Journal of Artificial Intelligence in Education, 26, 124–132.

    Article  Google Scholar 

  • Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005). AutoTutor: An Intelligent Tutoring System with mixed-initiative dialogue. IEEE Transactions in Education, 48, 612–618.

    Article  Google Scholar 

  • Graesser, A. C., Conley, M. W., & Olney, A. (2012a). Intelligent Tutoring Systems. In S. Graham & K. Harris (Eds.), APA handbook of educational psychology. Washington, DC: American Psychological Association.

    Google Scholar 

  • Graesser, A. C., & D’Mello, S. (2012). Emotions during the learning of difficult material. In B. Ross (Ed.), The psychology of learning and motivation (Vol. 57, pp. 183–225). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Graesser, A. C., D’Mello, S. K., Hu, X., Cai, Z., Olney, A., & Morgan, B. (2012b). AutoTutor. In P. M. McCarthy & C. Boonthum (Eds.), Applied natural language processing and content analysis: Identification, investigation and resolution (pp. 169–187). Hershey, PA: IGI Global.

    Google Scholar 

  • Graesser, A. C., D’Mello, S. K., Craig, S. D., Witherspoon, A. M., Sullins, J., McDaniel, B., et al. (2008). The relationship between affective states and dialogue patterns during interactions with AutoTutor. Journal of Interactive Learning Research, 19(2), 293–312.

    Google Scholar 

  • Graesser, A. C., Hu, X., Nye, B., & Sottilare, R. (2016). Intelligent Tutoring Systems, serious games, and the Generalized Intelligent Framework for Tutoring (GIFT). In H. F. O’Neil, E. L. Baker, & R. S. Perez (Eds.), Using games and simulation for teaching and assessment (pp. 58–79). Abingdon: Routledge.

    Google Scholar 

  • Graesser, A. C., Li, H., & Forsyth, C. (2014). Learning by communicating in natural language with conversational agents. Current Directions in Psychological Science, 23, 374–380.

    Article  Google Scholar 

  • Graesser, A. C., Lu, S., Jackson, G. T., Mitchell, H. H., Ventura, M., Olney, A. M., et al. (2004). AutoTutor: A tutor with dialogue in natural language. Behavioral, Research Methods, Instruction & Computers, 36, 180–193.

    Article  Google Scholar 

  • Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104–137.

    Article  Google Scholar 

  • Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9, 495–522.

    Article  Google Scholar 

  • Halpern, D. F., Millis, K., Graesser, A. C., Butler, H., Forsyth, C., & Cai, Z. (2012). Operation ARA: A computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills and Creativity, 7, 93–100.

    Article  Google Scholar 

  • Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90, 414–434.

    Article  Google Scholar 

  • Institute for Education Sciences. (2009). Science 2009: National assessment of educational progress at grades 4, 8, and 12. http://nces.ed.gov/nationsreportcard/pdf/main2009/2011451.pdf.

  • Jackson, G. T., & McNamara, D. S. (2013). Motivation and performance in a game-based Intelligent Tutoring System. Journal of Educational Psychology, 105, 1036–1049.

    Article  Google Scholar 

  • Johnson, L. W., & Valente, A. (2008). Tactical language and culture training systems: Using artificial intelligence to teach foreign languages and cultures. In M. Goker & K. Haigh (Eds.), Proceedings of the twentieth conference on innovative applications of artificial intelligence (pp. 1632–1639). Menlo Park, CA: AAAI Press.

    Google Scholar 

  • Koening, A. D. (2008). Exploring effective educational video game design: The interplay between narrative and game-schema construction (Unpublished doctoral dissertation). Arizona State University.

  • Landers, R. N. (2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752–768.

    Article  Google Scholar 

  • Landers, R. N., & Landers, A. K. (2014). An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming, 45(6), 769–785.

    Article  Google Scholar 

  • Lane, H. C., Noren, D., Auerbach, D., Birch, M., & Swartout, W. (2011). Intelligent tutoring goes to the museum in the big city: A pedagogical agent for informal science education. In G. Biswas, S. Bull, J. Kay, & A. Mitrovic (Eds.), Artificial intelligence in education: 15th International conference (pp. 155–162). Heidelberg: Springer.

  • Lepper, M. R., & Henderlong, J. (2000). Turning “play” into “work” and “work” into “play”: 25 years of research on intrinsic versus extrinsic motivation. In C. Sansone & J. M. Harackiewicz (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 257–307). San Diego, CA: Academic Press.

    Chapter  Google Scholar 

  • McNamara, D. S. (2004). SERT: Self-explanation reading training. Discourse Processes, 38, 1–30.

    Article  Google Scholar 

  • McNamara, D. S., Jackson, G. T., & Graesser, A. C. (2010). Intelligent tutoring and games (ITaG). In Y. K. Baek (Ed.), Gaming for classroom-based learning: Digital role-playing as a motivator of study (pp. 44–65). Hershey, PA: IGI Global.

    Chapter  Google Scholar 

  • McQuiggan, S. W., Robinson, J., & Lester, J. (2010). Affective transitions in narrative-centered learning environments. Educational Technology & Society, 13, 40–53.

    Google Scholar 

  • McQuiggan, S. W., Rowe, J. P., Lee, S., & Lester, J. C. (2008). Story-based learning: The impact of narrative on learning experiences and outcomes. Intelligent Tutoring Systems: Lecture Notes in Computer Science, 5091, 530–539.

    Article  Google Scholar 

  • Millis, K., Graesser, A., & Halpern, D. (2014). Operation ARA: A serious game that combines intelligent tutoring and learning principles to teach science. In V. Benassi, C. E. Overson, & C. M. Hakala, (Eds.) Applying the science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php.

  • National Science Foundation. (2012). National Science Board’s Science and Engineering Indicators 2012. http://www.nsf.gov/statistics/seind12/c7/c7h.htm.

  • Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24, 427–469.

    Article  Google Scholar 

  • Olney, A., D’Mello, S. K., Person, N., Cade, W., Hays, P., Williams, C., et al. (2012). Guru: A computer tutor that models expert human tutors. In S. Cerri, W. Clancey, G. Papadourakis, & K. Panourgia (Eds.), Proceedings of Intelligent Tutoring Systems (ITS) 2012 (pp. 256–261). Berlin: Springer.

    Google Scholar 

  • Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18, 315–341.

    Article  Google Scholar 

  • Richards, J., Stebbins, L., & Moellering, K. (2013). Games for a digital age: K-12 market map and investment analysis. New York: The Joan Ganz Cooney Center at Sesame Workshop.

    Google Scholar 

  • Ritterfeld, U., Cody, M., & Vorderer, P. (Eds.). (2009). Serious games: Mechanisms and effects. New York and London: Routledge, Taylor & Francis.

    Google Scholar 

  • Rowe, J., Shores, L. R., Mott, B., & Lester, J. (2011). Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education, 121, 115–133.

    Google Scholar 

  • Rus, V., D’Mello, S., Hu, X., & Graesser, A. C. (2013). Recent advances in intelligent systems with conversational dialogue. AI Magazine, 34, 42–54.

    Google Scholar 

  • Sabourin, J. L., Rowe, J. P., Mott, B. W., & Lester, J. C. (2013). Considering alternate futures to classify off-task behavior as emotion self-regulation: A supervised learning approach. Journal of Educational Data Mining, 5, 9–38.

    Google Scholar 

  • Sanchez, C. A., & Wiley, J. (2006). An examination of the seductive details effect in terms of working memory capacity. Memory & Cognition, 34, 344–355.

    Article  Google Scholar 

  • Shaffer, D. W. (2007). How computer games help children learn. New York, NY: Palgrave.

    Google Scholar 

  • Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  • Sweller, J. (1999). Instructional design in technical Areas. Camberwell, VIC: Australian Council for Educational Research.

    Google Scholar 

  • VanLehn, K. (2011). The relative effectiveness of human tutoring, Intelligent Tutoring Systems, and other tutoring systems. Educational Psychologist, 46, 197–221.

    Article  Google Scholar 

  • VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rose, P. A. (2007). When are tutorial dialogues more effective than reading? Cognitive Science, 30, 1–60.

    Article  Google Scholar 

  • Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34, 229–243.

    Article  Google Scholar 

  • Vygotsky, L. S. (1978). Mind in society: Development of higher psychological processes (p. 86). Cambridge: Harvard College.

    Google Scholar 

  • Wang, H., Shen, C., & Ritterfeld, U. (2009). Enjoyment of digital games: What makes them “seriously” fun? In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 25–47). New York and London: Routledge, Taylor & Francis.

    Google Scholar 

  • Woolf, B. P. (2009). Building Intelligent Tutoring Systems. Burlington, MA: Morgan Kaufman.

    Google Scholar 

  • Wouters, P., van Nimwegen, C., van der Spek, E. D., & van Oostendorp, H. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105, 249–265.

    Article  Google Scholar 

Download references

Acknowledgments

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305B070349 to Northern Illinois University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Author information

Authors and Affiliations

  1. Northern Illinois University, Dekalb, IL, USA

    Keith Millis, Patricia Wallace & Gary Timmins

  2. Educational Testing Service, Princeton, NJ, USA

    Carol Forsyth

  3. University of Memphis, Memphis, TN, USA

    Arthur C. Graesser

Authors
  1. Keith Millis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carol Forsyth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patricia Wallace
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arthur C. Graesser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gary Timmins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Keith Millis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Millis, K., Forsyth, C., Wallace, P. et al. The Impact of Game-Like Features on Learning from an Intelligent Tutoring System. Tech Know Learn 22, 1–22 (2017). https://doi.org/10.1007/s10758-016-9289-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-016-9289-5

Keywords

Search

Navigation