Skip to main content

Advertisement

SpringerLink
Log in

A solution-based intelligent tutoring system integrated with an online game-based formative assessment: development and evaluation

  • Development Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

Nowadays, intelligent tutoring systems are considered an effective research tool for learning systems and problem-solving skill improvement. Nonetheless, such individualized systems may cause students to lose learning motivation when interaction and timely guidance are lacking. In order to address this problem, a solution-based intelligent tutoring system (SITS) is integrated with an online game-based formative assessment game called tic-tac-toe quiz for single-player (TRIS-Q-SP) for learning computer programming. This assessment game combines tic-tac-toe with online assessment, and the rules of tic-tac-toe are revised to stimulate students to use online formative assessment actively. Finally, an experimental study is devised to assess the success of SITS, and significant achievements are observed for the experimental group, besides enjoyment and positive opinions toward the TRIS-Q-SP. Therefore, the practical use of SITS is supported, as the results indicate considerable advantages for the experimental group over the control group. The findings also reveal that immediate elaborated feedback upon answering each question in TRIS-Q-SP is part of an optimal design.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Albacete, P., & VanLehn, K. (2000). The conceptual helper: An intelligent tutoring system for teaching fundamental physics concepts. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent tutoring systems (Vol. 1839, pp. 564–573). Berlin: Springer.

    Chapter  Google Scholar 

  • Aldrich, C. (2009). Learning online with games, simulations, and virtual worlds. San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal ofthe Learning Sciences, 4(2), 167–207.

    Article  Google Scholar 

  • Azevedo, R., Guthrie, J. T., Wang, H., & Mulhern, J. (2001). Do different instructional interventions facilitate students ability to shift to more sophisticated mental models of complex systems?. Seattle, WA: The American Educational Research Association Annual Meeting.

    Google Scholar 

  • Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238. doi:10.3102/00346543061002213.

    Article  Google Scholar 

  • Bassat Levy, R., Ben-Ari, M., & Pekka, U. (2001). An extended experiment with Jelliot 2000. Paper presented at the in proceedings of the first international program visualization workshop, Porvoo—Finland, University of Joensuu.

  • Bentley, Y., Selassie, H., & Shegunshi, A. (2012). Design and evaluation of student-focused elearning. Electronic Journal of e-Learning, 10(1), 1–12.

    Google Scholar 

  • Bloom, B. S. (1984). The search for methods of group instruction as effective as one-to-one tutoring. Educational Leadership, 41(8), 4–17.

    Google Scholar 

  • Brown, K. G., Charlier, S. D., & Pierotti, A. (2012). E-Learning at work: Contributions of past research and suggestions for the future. International Review of Industrial and Organizational Psychology, 27, 89–114.

    Google Scholar 

  • Buchanan, T. (2000). The efficacy of a World-Wide Web mediated formative assessment. Journal of Computer Assisted learning, 16(3), 193–200. doi:10.1046/j.1365-2729.2000.00132.x.

    Article  Google Scholar 

  • Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. doi:10.3102/00346543065003245.

    Article  Google Scholar 

  • Butz, C. J., Hua, S., & Maguire, R. B. (2008). Web-based Bayesian intelligent tutoring systems. In R. Nayak, N. Ichalkaranje, & L. Jain (Eds.), Evolution of the web in artificial intelligence environments (Vol. 130, pp. 221–242). Berlin: Springer.

    Chapter  Google Scholar 

  • Byrne, B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications, and programming. New York: Routledge Academic.

    Google Scholar 

  • Conati, C., Gertner, A., & Vanlehn, K. (2002). Using Bayesian networks to manage uncertainty in student modeling. User Modeling and User-Adapted Interaction, 12(4), 371–417. doi:10.1023/a:1021258506583.

    Article  Google Scholar 

  • Corbett, A. T., & Anderson, J. R. (1989). Feedback timing and student control in the LISP intelligent tutoring system. Paper presented at the proceedings of the fourth international conference on artificial intelligence and education, Amsterdam, Netherlands.

  • Crowley, K., & Siegler, R. S. (1993). Flexible strategy use in young children’s tic-tac-toe. Cognitive Science, 17(4), 531–561. doi:10.1207/s15516709cog1704_3.

    Article  Google Scholar 

  • Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper & Row.

    Google Scholar 

  • Dalziel, J. (2001). Enhancing web-based learning with computer assisted assessment: pedagogical and technical considerations. In Proceedings of the fifth international computer assisted assessment conference. Loughborough: Loughborough University.

  • Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning: two case studies of Active Worlds as a medium for distance education. British Journal of Educational Technology, 36(3), 439–451. doi:10.1111/j.1467-8535.2005.00477.x.

    Article  Google Scholar 

  • Dihoff, R. E., Brosvic, G. M., Epstein, M. L., & Cook, M. J. (2003). The role of feedback during academic testing: The delay retention test revisited. The Psychological Record, 53, 533–548.

    Google Scholar 

  • Downs, E., & Shyam Sundar, S. (2011). “We won” vs. “They lost”: Exploring ego-enhancement and self-preservation tendencies in the context of video game play. Entertainment Computing, 2(1), 23–28. doi:10.1016/j.entcom.2011.03.012.

    Article  Google Scholar 

  • Eseryel, D., Ge, X., Ifenthaler, D., & Law, V. (2011). Dynamic modeling as a cognitive regulation scaffold for developing complex problem-solving skills in an educational massively multiplayer online game environment. Journal of Educational Computing Research, 45(3), 265–286. doi:10.2190/EC.45.3.a.

    Article  Google Scholar 

  • Fossati, D., Di Eugenio, B., Brown, C. W., Ohlsson, S., Cosejo, D. G., & Lin, C. (2009). Supporting computer science curriculum: Exploring and learning linked lists with ilist. Learning Technologies, IEEE Transactions on, 2(2), 107–120. doi:10.1109/TLT.2009.21.

    Article  Google Scholar 

  • Gálvez, J., Guzmán, E., & Conejo, R. (2009). A blended E-learning experience in a course of object oriented programming fundamentals. Knowledge-Based Systems, 22(4), 279–286. doi:10.1016/j.knosys.2009.01.004.

    Article  Google Scholar 

  • Gardner, L., Sheridan, D., & White, D. (2002). A web-based learning and assessment system to support flexible education. Journal of Computer Assisted learning, 18(2), 125–136. doi:10.1046/j.0266-4909.2001.00220.x.

    Article  Google Scholar 

  • Henly, D. C. (2003). Use of Web-based formative assessment to support student learning in a metabolism/nutrition unit. European Journal of Dental Education, 7(3), 116–122.

    Article  Google Scholar 

  • Holland, J., Mitrovic, A., & Martin, B. (2009). J-LATTE: A constraint-based tutor for Java. Paper presented at the proceedings of the 17th international conference on computers in education [CDROM], Hong Kong.

  • Hsiao, I. H., Sosnovsky, S., & Brusilovsky, P. (2010). Guiding students to the right questions: Adaptive navigation support in an E-Learning system for Java programming. Journal of Computer Assisted Learning, 26(4), 270–283. doi:10.1111/j.1365-2729.2010.00365.x.

    Article  Google Scholar 

  • Hung, K.-H. (2011). The design and development of an education-designed massively multiplayer online role-playing game (edd mmorpg) for young taiwanese mandarin-speaking learners learning english vocabulary words. New York: Columbia University Teacher’s College.

    Google Scholar 

  • Ifenthaler, D. (2008). Practical solutions for the diagnosis of progressing mental models. In D. Ifenthaler, P. Pirnay-Dummer, & J. M. Spector (Eds.), Understanding models for learning and instruction (pp. 43–61). New York: Springer.

    Chapter  Google Scholar 

  • Johnson, W. L. (1990). Understanding and debugging novice programs. Artificial Intelligence, 42(1), 51–97. doi:10.1016/0004-3702(90)90094-g.

    Article  Google Scholar 

  • Kulhavy, R. W., & Anderson, R. C. (1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 63(5), 505–512.

    Article  Google Scholar 

  • Lajoie, S. P. (2000). Computers as cognitive tools, volume II: No more walls. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Liu, H.-C., Andre, T., & Greenbowe, T. (2008). The impact of learner’s prior knowledge on their use of chemistry computer simulations: A case study. Journal of Science Education and Technology, 17(5), 466–482. doi:10.1007/s10956-008-9115-5.

    Article  Google Scholar 

  • Liu, C., Wang, S., & Zhang, C. (1999). Comments on the practice of developmental evaluation. Educational Research, 20(3), 75–76.

    Google Scholar 

  • McKimm, J., Jollie, C., & Cantillon, P. (2003). Web based learning. British Medical Journal (BMJ), 326(7394), 870–873.

    Article  Google Scholar 

  • Novak, J. D., & Gowin, D. B. (1984). Learning how to learn (1st ed.). New York: Cambridge University Press.

    Book  Google Scholar 

  • Pardos, Z., & Heffernan, N. (2010). Modeling Individualization in a Bayesian networks implementation of knowledge tracing. In P. De Bra, A. Kobsa, & D. Chin (Eds.), User modeling, adaptation, and personalization (Vol. 6075, pp. 255–266). Berlin: Springer.

    Chapter  Google Scholar 

  • Ramalingam, V., LaBelle, D., & Wiedenbeck, S. (2004). Self-efficacy and mental models in learning to program. SIGCSE Bulletin, 36(3), 171–175. doi:10.1145/1026487.1008042.

    Article  Google Scholar 

  • Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of games for educational purposes: A review of recent research. Simulation & Gaming, 23(3), 261–276. doi:10.1177/1046878192233001.

    Article  Google Scholar 

  • Seel, N. M., Ifenthaler, D., & Pirnay-Dummer, P. (2009). Mental models and problem solving: Technological solutions for measurement and assessment of the development of expertise. Model-based approaches to learning: Using systems models and simulations to improve understanding and problem solving in complex domains. Rotterdam: Sense Publishers.

  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. doi:10.3102/0034654307313795.

    Article  Google Scholar 

  • Shute, V. J., & Psotka, J. (1996). Intelligent tutoring systems: Past, present, and future. New York, NY: Macmillan.

    Google Scholar 

  • Spector, J. M. (2002). Knowledge management tools for instructional design. Educational Technology Research and Development, 50(4), 37–46. doi:10.1007/BF02504983.

    Article  Google Scholar 

  • Spector, J. M. (2012). Foundations of educational technology: Integrative approaches and interdisciplinary perspectives. New York, NY: Routledge.

    Google Scholar 

  • Spector, M., & Davidsen, P. I. (2000). Designing technology enhanced learning environments. Instructional and cognitive impacts of Web-based education, 241, 261.

    Google Scholar 

  • Spector, J. M., Merrill, M. D., Merrienboer, J. G., & Driscoll, M. P. (2008). Handbook of research on educational communications and technology (3rd ed.). New York: Routledge.

    Google Scholar 

  • Squire, K. D., DeVane, B., & Durga, S. (2008). Designing centers of expertise for academic learning through video games. Theory Into Practice, 47(3), 240–251.

    Article  Google Scholar 

  • Surber, J. R., & Anderson, R. C. (1975). Delay-retention effect in natural classroom settings. Journal of Educational Psychology, 67(2), 170–173.

    Article  Google Scholar 

  • Sykes, E. R. (2007). Developmental process model for the java intelligent tutoring system. Journal of Interactive Learning Research, 18(3), 399–410.

    Google Scholar 

  • Trevors, G., Duffy, M., & Azevedo, R. (2014). Note-taking within MetaTutor: Interactions between an intelligent tutoring system and prior knowledge on note-taking and learning. Educational Technology Research and Development, 62(5), 507–528. doi:10.1007/s11423-014-9343-8.

    Article  Google Scholar 

  • Tsai, F.-H., Kinzer, C., Hung, K.-H., Chen, C.-L. A., & Hsu, I. Y. (2013). The importance and use of targeted content knowledge with scaffolding aid in educational simulation games. Interactive Learning Environments, 21(2), 116–128. doi:10.1080/10494820.2012.705852.

    Article  Google Scholar 

  • Tsai, F.-H., Tsai, C.-C., & Lin, K.-Y. (2015). The evaluation of different gaming modes and feedback types on game-based formative assessment in an online learning environment. Computers & Education, 81, 259–269. doi:10.1016/j.compedu.2014.10.013.

    Article  Google Scholar 

  • Tsai, F. H., Yu, K. C., & Hsiao, H. S. (2012). Exploring the factors influencing learning effectiveness in digital game-based learning. Educational Technology & Society, 15(3), 240–250.

    Google Scholar 

  • van der Kleij, F. M., Timmers, C. F., & Eggen, T. J. H. M. (2011). The effectiveness of methods for providing written feedback through a computer-based assessment for learning: A systematic review. CADMO, 19(1), 21–39.

    Google Scholar 

  • Vasilyeva, E., Pechenizkiy, M., & De Bra, P. (2008). Adaptation of elaborated feedback in e-learning. In W. Nejdl, J. Kay, P. Pu, & E. Herder (Eds.), Adaptive hypermedia and adaptive web-based systems (Vol. 5149, pp. 235–244). Berlin: Springer.

    Chapter  Google Scholar 

  • Wang, T. H. (2007). What strategies are effective for formative assessment in an e-learning environment? Journal of Computer Assisted learning, 23(3), 171–186. doi:10.1111/j.1365-2729.2006.00211.x.

    Article  Google Scholar 

  • Wang, T. H. (2008). Web-based quiz-game-like formative assessment: Development and evaluation. Computers & Education, 51(3), 1247–1263. doi:10.1016/j.compedu.2007.11.011.

    Article  Google Scholar 

  • Weragama, D., & Reye, J. (2013). The PHP intelligent tutoring system. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Artificial intelligence in education (Vol. 7926, pp. 583–586). Berlin: Springer.

    Chapter  Google Scholar 

  • Westphal, B. T., Harris, F. C., Jr., & Fadali, M. S. (2003). Graphical programming: A vehicle for teaching computer problem solving. Paper presented at the in proceedings of the 33rd ASEE/IEEE frontiers in education conference, Boulder, CO, USA.

  • Winslow, L. E (1996). Programming pedagogy—A psychological overview. SIGCSE Bulletin, 28(3), 17–22. doi:10.1145/234867.234872.

    Article  Google Scholar 

  • Zapata-Rivera, D., & Greer, J. (2004). Interacting with Inspectable Bayesian student models. International Journal of Artificial Intelligence in Education, 14, 127–168.

    Google Scholar 

Download references

Acknowledgments

This research has been financially funded by the University of Malaya with project number of RG327-15AFR.

Author information

Authors and Affiliations

  1. Department of Software Engineering, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Danial Hooshyar, Rodina Binti Ahmad & Moein Fathi

  2. Center of Systems and Machines Intelligence, College of Engineering, Universiti Tenaga Nasional, 43300, Kajang, Malaysia

    Moslem Yousefi

  3. Malaysian Research Institute of Ageing (My Aging), Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia

    Abbas Abdollahi

  4. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Shi-Jinn Horng

  5. Department of Computer Science and Engineering, Korea University, Seoul, South Korea

    Heuiseok Lim

Authors
  1. Danial Hooshyar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodina Binti Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moslem Yousefi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Moein Fathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abbas Abdollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shi-Jinn Horng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Heuiseok Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Danial Hooshyar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hooshyar, D., Ahmad, R.B., Yousefi, M. et al. A solution-based intelligent tutoring system integrated with an online game-based formative assessment: development and evaluation. Education Tech Research Dev 64, 787–808 (2016). https://doi.org/10.1007/s11423-016-9433-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-016-9433-x

Keywords

Search

Navigation