Operation ARIES!: A Serious Game for Teaching Scientific Inquiry

  • Keith MillisEmail author
  • Carol Forsyth
  • Heather Butler
  • Patty Wallace
  • Arthur Graesser
  • Diane Halpern


Operation ARIES! is a serious game that teaches critical thinking about scientific inquiry. The player must help to identify aliens on Earth who are intentionally publishing bad research. The game combines aspects of video games and intelligent tutors in which the player holds conversations with animated agents using natural language. The player first takes a training course with a virtual trainee, followed by a module in which the player identifies flaws in research cases. In the third and final module, the player interviews suspected alien scientists on their research. Operation ARIES! is designed for high school seniors and adults.


Training Module Retrieval Practice Educational Game Story Line Human Player 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305B070349 to Northern Illinois University. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.


  1. Aleven, V., Koedinger, K.R.: An effective metacognitive strategy: Learning by doing and explaining with a computer-based cognitive tutor. Cogn. Sci. 26, 147–179 (2002)CrossRefGoogle Scholar
  2. Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: Lessons learned. J Learn. Sci. 4, 167–207 (1995)CrossRefGoogle Scholar
  3. Atkinson, R.K.: Optimizing learning from examples using animated pedagogical agents. J. Educ. Psychol. 94, 416–427 (2002)CrossRefGoogle Scholar
  4. Baylor, A.L., Kim, Y.: Simulating instructional roles through pedagogical agents. Int. J. Artif. Intell. Educ. 15, 95–115 (2005)Google Scholar
  5. Beck, I.L., McKeown, M.G., Hamilton, R.L., Kucan, L.: Questioning the Author: An Approach for Enhancing Student Engagement with Text. International Reading Association, Delaware (1997)Google Scholar
  6. Benyon, D., Turner, P., Turner, S.: Designing Interactive Systems. Addison-Wesley, Harlow (2005)Google Scholar
  7. Biswas, G., Leelawong, K., Schwartz, D., Vye, N., The Teachable Agents Group at Vanderbilt.: Learning by teaching: A new agent paradigm for educational software. Appl. Artif. Intell. 19, 363–392 (2005)CrossRefGoogle Scholar
  8. Biswas, G., Jeong, H., Kinnebrew, J., Sulcer, B., & Roscoe, R.: Measuring self-regulated learning skills through social interactions in a Teachable Agent environment. Res. Pract. Technol.-Enhanced Learn. 5, 123–152 (2010)CrossRefGoogle Scholar
  9. Bloom, B.S.: Taxonomy of Educational Objectives, the Classification of Educational Goals – Handbook I: Cognitive Domain. McKay, New York (1956)Google Scholar
  10. Bransford, J.D., Sherwood, R.S., Hasselbring, T.S., Kinzer, C.K., Williams, S.M.: Anchored instruction: Why we need it and how technology can help. In: Nix, D., Spiro, R. (eds.) Cognition, Education, and Multimedia: Exploring Ideas in High Technology, pp. 115–141. Lawrence Erlbaum Associates, Hillsdale, NJ (1990)Google Scholar
  11. Bryant, J., Fondren, W.: Psychological and communicological theories of learning and emotion underlying serious games. In: Ritterfeld, U., Cody, M., Vorderer, P. (eds.) Serious Games: Mechanisms and Effects, pp. 103–116. Routledge, Taylor & Francis, New York and London (2009)Google Scholar
  12. Chi, M.T.H., Siler, S.A., Jeong, H., Yamauchi, T., Hausmann, R.G.: Learning from human tutoring. Cogn. Sci. 25, 471–553 (2001)CrossRefGoogle Scholar
  13. Craig, S.D., Sullins, J., Witherspoon, A., Gholson, B.: Deep level reasoning questions effect: The role of dialog and deep-level reasoning questions during vicarious learning. Cogn. Instruct. 24, 563–589 (2006)CrossRefGoogle Scholar
  14. Craig, S.D., Chi, M.T.H., VanLehn, K.: Improving classroom learning by collaboratively observing human tutoring videos while problem solving. J. Educ. Psychol. 101, 779–789 (2009)CrossRefGoogle Scholar
  15. Csikszenthimhalyi, M.: Flow: The Psychology of Happiness. Random House, London (2002)Google Scholar
  16. D’Mello, S., Graesser, A.C.: Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model.User-adapted Interact. 20, 147–187 (2010)CrossRefGoogle Scholar
  17. Gee, J.P.: What Video Games Teach Us About Language and Literacy. Palgrave/Macmillan, New York (2003)Google Scholar
  18. Graesser, A.C., Person, N.K., & Magliano, J.P.:  Collaborative dialogue patterns in naturalistic one-to-one tutoring.  Appl. Cogn. Psychol. 9, 359.1–359.28 (1995)CrossRefGoogle Scholar
  19. Graesser, A.C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R., the TRG: Auto Tutor: A simulation of a human tutor. J. Cogn. Syst. Res. 1, 35–51 (1999)CrossRefGoogle Scholar
  20. Graesser, A.C., Person, N., Harter, D., the Tutoring Research Group: Teaching tactics and dialog in AutoTutor. Int. J. Artif. Intell. Educ. 12, 257–279 (2001)Google Scholar
  21. Graesser, A.C., Lu, S., Jackson, G.T., Mitchell, H., Ventura, M., Olney, A., Louwerse, M.M.: AutoTutor: A tutor with dialogue in natural language. Behav. Res. Meth. Instrum. Comput. 36, 180–193 (2004)CrossRefGoogle Scholar
  22. Graesser, A.C., Chipman, P., Haynes, B.C., Olney, A.: AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Trans. Educ. 48, 612–618 (2005a)CrossRefGoogle Scholar
  23. Graesser, A.C., Lu, S., Olde, B.A., Cooper-Pye, E., Whitten, S.: Question asking and eye tracking during cognitive disequilibrium: comprehending illustrated texts on devices when the devices break down. Mem. Cognit. 33, 1235–1247 (2005b)Google Scholar
  24. Graesser, A.C., McNamara, D.S., VanLehn, K.: Scaffolding deep comprehension strategies through Point&Query, AutoTutor, and iSTART. Educ. Psychol. 40, 225–234 (2005c)Google Scholar
  25. Graesser, A.C., D’Mello, S.K., Craig, S.D., Witherspoon, A., Sullins, J., McDaniel, B., Gholson, B.: The relationship between affective states and dialog patterns during interactions with AutoTutor. J. Interact. Learn. Res. 19, 293–312 (2008)Google Scholar
  26. Graesser, A.C., Chipman, P., Leeming, F., Biedenbach, S.: Deep learning and emotion in serious games. In: Ritterfeld, U., Cody, M., Vorderer, P. (eds.) Serious Games: Mechanisms and Effects, pp. 83–102. Routledge, Taylor & Francis, New York and London (2009a)Google Scholar
  27. Graesser, A., Ozuru, Y., Sullins, J.: What is a good question? In: McKeown, M.G., Kucan, L. (eds.) Threads of Coherence in Research on the Development of Reading Ability, pp. 112–141. Guilford, New York (2009b)Google Scholar
  28. Griffin, T.D., Wiley, J., Thiede, K.W.: Individual differences, rereading, and self-explanation: Concurrent processing and cue validity as constraints on metacomprehension accuracy. Mem. Cogn. 36, 93–103 (2008)CrossRefGoogle Scholar
  29. Halpern, D.F.: Teaching for critical thinking: A four-part model to enhance thinking skills. In: Davis, S., Buskist, W. (eds.) The Teaching of Psychology: Essays in Honor of Wilbert J. McKeachie and Charles L. Brewer, pp. 91–105. Lawrence Erlbaum, Mahwah, NJ (2002)Google Scholar
  30. Harackiewicz, J.: The effects of reward contingency and performance feedback on intrinsic motivation. J. Pers. Soc. Psychol. 37, 1352–1363 (1979)CrossRefGoogle Scholar
  31. Hartmann, T.: Gender differences in the use of computer-games as competitive leisure activities. Paper presented at Digital Games Research Association (DIGRA), November 4–6, 2003, Utrecht, The Netherlands (2003)Google Scholar
  32. Heeter, C., Egidio, R., Punya, M., Winn, B., Caywood, J.: Alien games: Do girls prefer games designed by girls? Games Cult. 4(1), 74–100 (2007). doi: 10.1177/1555412008325481CrossRefGoogle Scholar
  33. Hynd, C., Alverman, D.E.: Overcoming misconceptions in science: An on-line study of prior knowledge activation. Reading Res. Instr. 84, 12–26 (1989)CrossRefGoogle Scholar
  34. Indian Wine Academy: Study finds wine lowers heart attack risk in women. (2007)
  35. Ketelhut, D.J.: The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment.  J. Sci. Educ. Technol. 16, 99–111 (2007)CrossRefGoogle Scholar
  36. King, A.: Guiding knowledge construction in the classroom: Effects of teaching children how to question and how to explain. Am. Educ. Res. J. 31, 338–368 (1994)Google Scholar
  37. Kopp, K., Britt, A., Millis, K., Graesser, A.: Improving the efficiency of dialogue in tutoring. J. Learn. Instr. (in press)Google Scholar
  38. Kulik, J.A., Kulik, C-L.C.: Timing of feedback and verbal learning. Rev. Educ. Res. 58, 79–97 (1988)MathSciNetGoogle Scholar
  39. Landauer, T.K., & Dumais, S.T.: A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104, 211–240 (1997)CrossRefGoogle Scholar
  40. Linn, M.C., Davis, E.A., & Bell, P.: Internet Environments for Science Education. Erlbaum, Hillsdale, New Jersey (2004)Google Scholar
  41. Maki, R.H.: Test predictions over text material. In: Hacker, D.J., Dunlosky, J., Graesser, A.C. (eds.) Metacognition in Educational Theory and Practice, pp. 117–144. Erlbaum, Mahwah, NJ (1998)Google Scholar
  42. Malone, T.W., Lepper, M.R.: Making learning fun: A taxonomy of intrinsic motivations for learning. In: Snow, R.E., Farr, M.J. (eds.) Aptitude Learning and Instruction, Vol. 3, pp. 223–253. Erlbaum, Hillsdale, NJ (1987)Google Scholar
  43. McNamara, D.S., O’Reilly, T., Rowe, M., Boonthum, C., Levinstein, I.: iSTART: A web-based tutor that teaches self-explanation and metacognitive reading strategies. In: McNamara, D.S. (ed.) Reading Comprehension Strategies: Theories, Interventions, and Technologies, pp. 397–420. Erlbaum, New York (2007)Google Scholar
  44. Meyer, B.J.F., Wijekumar, K.: A web-based tutoring system for the structure strategy: Theoretical background, design, and findings. In: McNamara, D.S. (ed.) Reading Comprehension Strategies: Theories, Interventions, and Technologies, pp. 347–374. Erlbaum, New York (2007)Google Scholar
  45. National Center for Complementary and Alternative Medicine: Homeopathy: An introduction. Retrieved from the National Institutes of Health website (undated)
  46. National Science Education Standards (NSES): The National Academies Press, Washington, DC (1996)Google Scholar
  47. Oxland, K.: Gameplay and Design. Addison-Wesley, Harlow (2004)Google Scholar
  48. Palincsar, A.S., Brown, A.L.: Reciprocal teaching of comprehension-fostering and comprehension-monitoring Activities. Cogn. Instr. 1, 117–175 (1984)Google Scholar
  49. Ratan, R., Ritterfeld, U.: Classifying serious games. In: Ritterfeld, U., Cody, M., Vorderer, P. (eds.) Serious Games: Mechanisms and Effects, pp. 10–24. Routledge, New York (2009)Google Scholar
  50. Rieber, L.: Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educ. Technol. Res. Dev. 44, 42–58 (1996)Google Scholar
  51. Roediger, H.L., Karpicke, J.D.: Test-enhanced learning: Taking memory tests improves long-term retention. Psychol. Sci., 17, 249–255 (2006)CrossRefGoogle Scholar
  52. Rosenshine, B., Meister, C., Chapman, S.: Teaching students to generate questions: A review of the intervention studies. Rev. Educ. Res., 66, 181–221 (1996)Google Scholar
  53. Salen, K., Zimmerman, E.: Rules of Play: Game Design and Fundamentals. The MIT Press, Cambridge, MA (2004)Google Scholar
  54. Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78, 153–189 (2008)CrossRefGoogle Scholar
  55. Taboada, A., Guthrie, J.T.: Contributions of student questioning and prior knowledge to construction of knowledge from reading information text. J. Literacy Res. 38, 1–35 (2006)CrossRefGoogle Scholar
  56. Van Eck, R.: Building artificially intelligent learning games. In: Gibson, D., Aldrich, C., Prensky, M. (eds.) Games and Simulations in Online Learning Research & Development Frameworks, pp. 271–307. Idea Group, Hershey, PA (2007)Google Scholar
  57. VanLehn, K., Graesser, A.C., Jackson, G.T., Jordan, P., Olney, A., Rose, C.P.: When are tutorial dialogues more effective than reading? Cogn. Sci. 31, 3–62 (2007)Google Scholar
  58. Vygotsky, L.: Mind in Society: The Development of Higher Psychological Functions. Harvard University Press, Cambridge, MA (1978)Google Scholar
  59. Wang, H., Shen, C., & Ritterfeld, U.: Enjoyment of digital games: What makes them “seriously” fun? In: Ritterfeld, U., Cody, M., Vorderer, P. (eds.) Serious Games: Mechanisms And Effects, pp. 25–47. Routledge, Taylor & Francis, New York and London (2009)Google Scholar
  60. Whitton, N.: Learning with Digital Games: A Practical Guide to Engaging Students in Higher Education. Routledge, New York (2010)Google Scholar
  61. Wisher, R.A., Graesser, A.C.: Question asking in advanced distributed learning environments. In: Fiore, S.M., Salas, E. (eds.) Toward a Science of Distributed Learning and Training, pp. 209–234. American Psychological Association, Washington, DC (2007)Google Scholar
  62. Yee, N.: Motivations of play in online games. CyberPsychol. Behav. 9, 772–775 (2006)CrossRefGoogle Scholar
  63. Yeomans, J.: Dynamic assessment practice: Some suggestions for ensuring follow up. Educ. Psychol. Pract. 24, 105–114 (2008)CrossRefGoogle Scholar


Key Books

  1. Gee, J. (ed.): Games, Learning, Assessment. MIT Press, Boston, MA (in press)Google Scholar
  2. Gibson, D., Aldrich, C., Prensky, M. (eds.): Games and Simulations in Online Learning: Research and Development Frameworks. Information Science Publishing, Hershey, PA (2006)Google Scholar
  3. Halpern, D.F. (ed.): Undergraduate Education in Psychology: A Blueprint for the Future of the Discipline. American Psychological Association Books, Washington, DC (2010)Google Scholar
  4. Mayer, R.E., Alexander, P.A. (eds.): Handbook of Research on Learning and Instruction. Routledge Press, New York, NY (2011)Google Scholar
  5. Mayrath, M., Robisnon, D., Clarke-Midura, J. (eds.): Technology-Based Assessments for 21st Century Skills: Theoretical and Practical Implications from Modern Research. Information Age Publications, Charlotte, NC (2011)Google Scholar
  6. Ritterfeld, U., Cody, M., Vorderer, P. (eds.): Serious Games: Mechanisms and Effects. Routledge, Taylor and Francis, Mahwah, NJ (2009)Google Scholar
  7. Shaffer, D.W.: How Computer Games Help Children Learn. Palgrave Macmillan, New York (2006)CrossRefGoogle Scholar
  8. Spector, J.M., Merrill, M.D., van Merriënboer, J.J.G., Driscoll, M.P. (eds.): Handbook of Research on Educational Communications and Technology. Taylor & Francis, London (2008)Google Scholar
  9. Van Eck, R.: Interdisciplinary Models and Tools for Serious Games: Emerging Concepts and Future Directions. Information Science Reference, Hershey, NY (2010)Google Scholar

Key Survey/Review Articles

  1. Aleven, V.: An intelligent learning environment for case-based argumentation. Technol. Inst. Cogn. Lear. 4(2), 191–241 (2006)Google Scholar
  2. Bagley, E., Shaffer, D.W.: When people get in the way: Promoting civic thinking through epistemic gameplay. Int. J. Gaming Comput. -Mediated Simul. 1, 36–52 (2009)CrossRefGoogle Scholar
  3. Bråten, I., Strømsø, H.I., Britt, M.A.: Trust matters: Examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Res. Q. 44, 6–28 (2009)CrossRefGoogle Scholar
  4. Gee, J.P.: Why game studies now? E-video games: a new art form. Games Cult. 1, 1–4 (2006)Google Scholar
  5. Graesser, A.C., Jeon, M., Dufty, D.: Agent technologies designed to facilitate interactive knowledge construction. Discourse Process. 45, 298–322 (2008)CrossRefGoogle Scholar
  6. Millis, K., Magliano, J., Todaro, S.: Measuring discourse-level processes with verbal protocols and latent semantic analysis. Sci. Stud. Reading 10, 225–240 (2006)CrossRefGoogle Scholar
  7. Nash, P., Shaffer, D.W.: Mentor modeling: The internalization of modeled professional thinking in an epistemic game. J. Comput. Assis. Lear. 27, 173–189. (2011)CrossRefGoogle Scholar
  8. Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78, 153–189 (2008)CrossRefGoogle Scholar
  9. VanLehn, K., Graesser, A.C., Jackson, G.T., Jordan, P., Olney, A.M., Rose, C.: When are tutorial dialogues more effective than reading? Cogn. Sci. 31, 3–62 (2007)Google Scholar
  10. Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, C.A., Ash, I.K., Hemmerich, J.A.: Source evaluation, comprehension, and learning in internet science inquiry tasks. Am. Educ. Res. J. 46, 1060–1106 (2009)CrossRefGoogle Scholar

Organisations, Societies, Special Interest Groups

Research Groups

  1. The Discourse and Technology Group at Northern Illinois University
  2. The Institute for Intelligent Systems at The University of Memphis http://

Key International Conferences/Workshops

  1. AIED: 15th International Conference on Artificial Intelligence in Education (2011)
  2. ITS: Tenth International Conference on Intelligent Tutoring Systems: Bridges to Learning (2010)
  3. ST&D: Twenty-first Annual Meeting of the Society for Text and Discourse (2011)

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Keith Millis
    • 1
    Email author
  • Carol Forsyth
    • 2
  • Heather Butler
    • 3
  • Patty Wallace
    • 4
  • Arthur Graesser
    • 2
  • Diane Halpern
    • 5
  1. 1.Department of PsychologyNorthern Illinois UniversityDeKalbUSA
  2. 2.University of MemphisMemphisUSA
  3. 3.Claremont Graduate UniversityClaremontUSA
  4. 4.Northern Illinois UniversityDeKalbUSA
  5. 5.Claremont McKenna CollegeClaremontUSA

Personalised recommendations