Education and Information Technologies

, Volume 23, Issue 6, pp 2423–2452 | Cite as

The effect of simulation games on learning computer programming: A comparative study on high school students’ learning performance by assessing computational problem-solving strategies

  • Nikolaos PellasEmail author
  • Spyridon Vosinakis


Computer games are quickly gaining momentum by enabling new approaches to teaching and learning experience for programming courses in K-12 curriculum. However, it remains unclear if the game interface and elements created by using three-dimensional (3D) virtual worlds combined with visual programming languages or a visual programming environment can affect students’ learning performance. This quasi-experimental study presents evidence about how a game can assist boys and girls to gain a greater understanding on skills related to CT for developing, implementing and transforming their solution plans into code based on their computational problem-solving strategies. A total of fifty (n = 50) high school students who volunteered to participate in this study divided into a control group (n = 25) and an experimental (n = 25) group that used Scratch and OpenSim with the Scratch4SL palette, respectively to propose their solutions for the same problem-solving tasks via a simulation game. The study findings indicate substantial differences and important points of view about students’ learning performance by assessing their computational problem-solving strategies. Students from the experimental group performed significantly better both in measures of problem-solving and algorithmic thinking. Mean scores on post-questionnaires from the experimental group revealed improvements higher than their control group counterparts in two aspects. First, students of the former group created more complete computational instructions with rules to be specified and delivered the learning goals. Second, students of the same group proposed and applied more correct computational concepts and practices in code. Finally, this study discusses the implications for designing learning experiences using OpenSim with Scratch4SL.


Computational problem-solving Computer programming Scratch 3D virtual worlds 


Compliance with ethical standards

Conflict of interest

All authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


  1. ACM Education Policy Committee. (2014). Rebooting the pathway to success: Preparing students for computing workforce needs in the United States. Retrieved 20 March 2016 from:
  2. Bienkowski, M., Snow, E., Rutstein, D. W., & Grover, S. (2015). Assessment design patterns for computational thinking practices in secondary computer science: A first look (SRI technical report). Menlo Park: SRI International.Google Scholar
  3. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 Annual meeting of the American Educational Research Association, Vancouver.Google Scholar
  4. Chao, P.-Y. (2016). Exploring students’ computational practice, design and performance of problem-solving through a visual programming environment. Computers & Education, 95(2), 202–215.CrossRefGoogle Scholar
  5. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education. Abingdon: Routledge.Google Scholar
  6. CSTA & ISTE (2011). Computational thinking in K–12 education leadership toolkit. Retrieved 20 March 2016 from:
  7. Dalgarno, B., & Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1), 10–32.CrossRefGoogle Scholar
  8. Davies, S. (2008). The effects of emphasizing computational thinking in an introductory programming course. Frontiers in Education Conference (FIE 2008). 38th Annual, IEEE.
  9. de Raadt, M. (2007). A review of Australasian investigations into problem-solving and the novice programmer. Computer Science Education, 17(3), 201–213.CrossRefGoogle Scholar
  10. Denner, J., Werner, L., & Ortiz, E. (2012). Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers & Education, 58(1), 240–249.CrossRefGoogle Scholar
  11. Esteves, M., Fonseca, B., Morgado, L., & Martins, P. (2011). Improving teaching and learning of computer programming through the use of the second life virtual world. British Journal of Educational Technology, 42(4), 624–637.CrossRefGoogle Scholar
  12. Garneli, V., & Chorianopoulos, K. (2017). Programming video games and simulations in science education: Exploring computational thinking through code analysis. Interactive Learning Environments, 26, 386–401. Scholar
  13. Garneli, V., Giannakos, M., & Chorianopoulos, K. (2015). Computing education in K-12 schools: A review of the literature. IEEE Global Engineering Education Conference (EDUCON) (pp. 536-544). IEEE: Tallinn, Estonia.Google Scholar
  14. Girvan, C., Tangney, B., & Savage, T. (2013). SLurtles: Supporting constructionist learning in 'Second Life'. Computers & Education, 61(4), 115–132.CrossRefGoogle Scholar
  15. Good, J., Howland, K., & Thackray, L. (2008). Problem-based learning spanning real and virtual worlds: A case study in second life. ALT-J, Research in Learning Technology, 16(3), 163–172.CrossRefGoogle Scholar
  16. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.CrossRefGoogle Scholar
  17. Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science course for middle school students. Computers Science Education, 25(2), 199–237.CrossRefGoogle Scholar
  18. Howland, K., & Good, J. (2015). Learning to communicate computationally with Flip: A bi-modal programming language for game creation. Computers & Education, 80(2), 224–240.CrossRefGoogle Scholar
  19. Jakos, F., & Verber, D. (2016). Learning basic programming skills with educational games: A case of primary schools in Slovenia. Journal of Educational Computing Research, 55, 673–698. Scholar
  20. Kafai, Y., & Burke, Q. (2015). Constructionist gaming: Understanding the benefits of making games for learning. Educational Psychologist, 50(4), 313–334.CrossRefGoogle Scholar
  21. Knight, J. K. (2010). Biology concept assessment tools: Design and use. Microbiology, 5.Google Scholar
  22. Koorsse, M., Cilliers, C., & Calitz, A. (2015). Programming assistance tools to support the learning of IT programming in south African secondary schools. Computers & Education, 82(2), 162–178.CrossRefGoogle Scholar
  23. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the Computational Thinking Scales (CTS). Computers in Human Behavior, 72, 558–569. Scholar
  24. Lahtinen, E., Ala-Mutka, K., & Järvinen, H. (2005). A study of the difficulties of novice programmers. In: Proceedings of the 10th Annual SIGCSE Conference on innovation and Technology in Computer Science Education (pp. 14–18). Caparica: ACM.Google Scholar
  25. Liu, C. C., Cheng, Y. B., & Huang, C. W. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57, 1907–1918.CrossRefGoogle Scholar
  26. Liu, Z., Zhi, R., Hicks, Z., & Barnes, T. (2017). Understanding problem solving behavior of 6–8 graders in a debugging game. Computer Science Education, 27, 1–29. Scholar
  27. Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41(3), 51–61.CrossRefGoogle Scholar
  28. Marx, J. D., & Cummings, K. (2007). Normalized change. American Journal of Physics, 75, 87–91.CrossRefGoogle Scholar
  29. Mouza, C., Marzocchi, A., Pan, Y., & Pollock, L. (2016). Development, implementation, and outcomes of an equitable computer science after-school program: Findings from middle-school students. Journal of Research on Technology in Education, 48(2), 84–104.CrossRefGoogle Scholar
  30. Pellas, N., & Vosinakis, S. (2017a). How can a simulation game support the development of computational problem-solving strategies? In IEEE Global Engineering Education Conference (pp. 1124–1131). IEEE: Greece, Athens.Google Scholar
  31. Pellas, N., & Vosinakis, S. (2017b). Learning to think and practice computationally via a 3D simulation game. 11th International Conference on Interactive Mobile Communication, Technologies and Learning (IMCL 2017). In IMCL2017 Proceedings "Advances in Intelligent Systems and Computing" (pp. 193–204). M. E. Auer and T. Tsiatsos (Eds.): IMCL 2017, AISC 725, (pp. 550–562). Thessaloniki, Greece: Springer.Google Scholar
  32. Repenning, A., Webb, D., & Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. In Proceedings of the 41st ACM Technical Symposium On Computer Science Education (SIGCSE ‘10), (pp. 265–269). New York, NY: ACM Press.Google Scholar
  33. Rico, M., Martνnez-Muρoz, G., Alaman, X., Camacho, D., & Pulido, E. (2011). Improving the programming experience of high school students by means of virtual worlds. International Journal of Engineering Education, 27(1), 52–60.Google Scholar
  34. Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13, 137–172.CrossRefGoogle Scholar
  35. Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2016). Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior, 72(1), 678–691.Google Scholar
  36. Rosenbaum, E. (2008). Scratch for second life. In S. Veeragoudar Harrell (Chair & Organizer), Virtually there: Emerging designs for STEM teaching and learning in immersive online 3D microworlds. Symposium in proceedings of the international conference on learning sciences – ICLS 2008. Utrecht, The Netherlands: ICLS.Google Scholar
  37. Singh, K. (2007). Quantitative social research methods. Thousand Oaks: Sage Publications.CrossRefGoogle Scholar
  38. Slavin, R. E., Cheung, A., Groff, C., & Lake, C. (2007). Effective reading programs for middle and high schools: A best-evidence synthesis. Baltimore: Johns Hopkins University, Center for Data-Driven Reform in Education.Google Scholar
  39. Topu, F. B., Reisoğlu, İ., Yılmaz, T. K., et al. (2018). Information retention’s relationships with flow, presence and engagement in guided 3D virtual environments. Education and Information Technologies. Scholar
  40. Webb, H., & Rosson, M. B. (2013). Using scaffolded examples to teach computational thinking concepts. In Proceeding of the 44th ACM technical symposium on computer science education (pp. 95–100). ACM.Google Scholar
  41. Werner, L., Denner, J., & Campe, S. (2015). Children programming games: A strategy for measuring computational learning. ACM Transactions on Computing Education, 14, 24.Google Scholar
  42. Wing, J. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.CrossRefGoogle Scholar
  43. Witherspoon, E. B., Higashi, R. M., Schunn, C. D., Baehr, E. C., & Shoop, R. (2017). Developing computational thinking practices through a virtual robotics programming curriculum. ACM Transactions on Computing Education, 18(1), 20.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Product and Systems Design EngineeringUniversity of the AegeanHermoupolisGreece

Personalised recommendations