Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development

Abstract

Underrepresentation of women in computer science (CS) increasingly demands the necessity to find and enhance current learning engagement approaches to bring more women into computing fields. Some researchers have been exploring the influence of gamification on female students as one of these possible learning engagement strategies. Gamification refers to the introduction of video game elements into non-game activities to enhance engagement and motivation. Previous studies have reported mixed results of the impact of gamification on women. In this study, we introduce SEP-CyLE (Software Engineering and Programming Cyberlearning Environment), an online gamified tool that was designed to provide supplemental computing content to students. This paper presents a convergent mixed-methods study guided by social identity theory and self-efficacy to understand women's experiences with this gamified tool. More specifically, this study explores virtual points' and leaderboards' effects on CS identity development, self-efficacy, and performance. The results show that virtual points and the leaderboard contributed to improved performance for students of all genders, suggesting that gamification is a gender-neutral learning engagement strategy that improves female students' performance as much as male students. Regardless of improved performance, most women did not actively enjoy or were motivated by the virtual points or leaderboard in SEP-CyLE. Additionally, gamification had no significant impact on CS identity development or self-efficacy constructs and had little to no impact on women's interest and engagement in the field of computing.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Adams, W. C. (2015). Conducting semi-structured interviews. In Handbook of practical program evaluation (4th ed., pp. 492–505). https://doi.org/https://doi.org/10.1002/9781119171386.ch19

  2. Aldemir, T., Celik, B., & Kaplan, G. (2018). A qualitative investigation of student perceptions of game elements in a gamified course. Computers in Human Behavior, 78, 235–254.

    Article  Google Scholar 

  3. Alomari, H. W., Ramasamy, V., Kiper, J. D., & Potvin, G. (2020). A User Interface (UI) and User eXperience (UX) evaluation framework for cyberlearning environments in computer science and software engineering education. Heliyon, 6(5), e03917.

    Article  Google Scholar 

  4. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological review, 84(2), 191–215.

    Article  Google Scholar 

  5. Banfield, J., & Wilkerson, B. (2014). Motivation and self-efficacy. Contemporary Issues in Education Research, 7(4), 291–298.

    Article  Google Scholar 

  6. Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD research, 1(1), 19.

    Google Scholar 

  7. Bernik, A., Bubas, G., & Radosevic, D. (2015). A pilot study of the influence of gamification on the effectiveness of an e-learning course. In Central European conference on information and intelligent systems (p. 73). Faculty of Organization and Informatics Varazdin.

  8. Bista, S. K., Nepal, S., Colineau, N., & Paris, C. (2012). Using gamification in an online community. CollaborateCom, 2012, 611–618.

    Google Scholar 

  9. Bogost, I. (2011). How to do things with videogames. U of Minnesota Press.

  10. Boumi, S., & Vela, A. (2019). Application of hidden Markov models to quantify the impact of enrollment patterns on student performance. In International Educational Data Mining Society.

  11. Boumi, S., Vela, A., & Chini, J. (2020). Quantifying the relationship between student enrollment patterns and student performance. arXiv preprint .

  12. Brickhouse, N. W., & Potter, J. T. (2001). Young women’s scientific identity formation in an urban context. Journal of Research in Science Teaching, 38(8), 965–980.

    Article  Google Scholar 

  13. Brickhouse, N. W., Lowery, P., & Schultz, K. (2000). What kind of girl does science? The construction of school science identities. Journal of Research in Science Teaching, 37(5), 441–458. https://doi.org/10.1002/(SICI)1098-2736(200005)37:5%3c441::AID-TEA4%3e3.0.CO;2-3

    Article  Google Scholar 

  14. Buffum, P. S. (2015). Leveraging narrative interactivity to foster computer science identities. Proceedings of the 11th ACM Conference on International Computing Education Research—ICER '15, 247–248. doi: https://doi.org/10.1145/2787622.2787734

  15. Buisman, A. L., & van Eekelen, M. C. (2014, November). Gamification in educational software development. In Proceedings of the Computer Science Education Research Conference (pp. 9–20). ACM.

  16. Cafazzo, J. A., Casselman, M., Hamming, N., Katzman, D. K., & Palmert, M. R. (2012). Design of a mHealth app for the self-management of adolescent type 1 diabetes: A pilot study. Journal of medical Internet research, 14(3), e70.

    Article  Google Scholar 

  17. Çakır, N. A., Gass, A., Foster, A., & Lee, F. J. (2017). Development of a game-design workshop to promote young girls’ interest towards computing through identity exploration. Computers and Education, 108, 115–130.

    Article  Google Scholar 

  18. Camilleri, V., Busuttil, L., & Montebello, M. (2011). Social interactive learning in multiplayer games. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), Serious games and edutainment applications (pp. 481e501). London, England: Springer-Verlag.

  19. Carbonaro, M., Szafron, D., Cutumisu, M., & Schaeffer, J. (2010). Computer-game construction: A gender-neutral attractor to Computing Science. Computers and Education, 55(3), 1098–1111.

    Article  Google Scholar 

  20. Chang-lau, R. & Clarke, P. J. (2018). Software engineering and programming cyberlearning environment (SEP-CyLE), https://stem-cyle.cis.fiu.edu/.

  21. Cheryan, S., Meltzoff, A. N., & Kim, S. (2011). Classrooms matter: The design of virtual classrooms influences gender disparities in computer science classes. Computers and Education, 57(2), 1825–1835.

    Article  Google Scholar 

  22. Cheryan, S., Plaut, V. C., Davies, P. G., & Steele, C. M. (2009). Ambient belonging: How stereotypical cues impact gender participation in computer science. Journal of Personality and Social Psychology, 97(6), 1045–1060. https://doi.org/10.1037/a0016239

    Article  Google Scholar 

  23. Christy, K. R., & Fox, J. (2014). Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Computers and Education, 78, 66–77.

    Article  Google Scholar 

  24. Cramer, H., Rost, M., & Holmquist, L. E. (2011, August). Performing a check-in: Emerging practices, norms and ‘conflicts' in location-sharing using Foursquare. In Proceedings of the 13th international conference on human-computer interaction with mobile devices and services (pp. 57–66). ACM.

  25. Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks: Sage publications.

    Google Scholar 

  26. Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks: Sage publications.

    Google Scholar 

  27. Cribbs, J. D., Hazari, Z., Sonnert, G., & Sadler, P. M. (2015). Establishing an explanatory model for mathematics identity. Child development, 86(4), 1048–1062.

    Article  Google Scholar 

  28. De Almeida Souza, M. R., Constantino, K. F., Veado, L. F., & Figueiredo, E. M. L. (2017, November). Gamification in software engineering education: An empirical study. In Software Engineering Education and Training (CSEE&T). In 2017 IEEE 30th Conference on IEEE (pp. 276–284).

  29. De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning. Computers and Education, 75, 82–91.

    Article  Google Scholar 

  30. Denny, P. (2013, April). The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 763–772). ACM.

  31. Deterding, S., Sicart, M., Nacke, L., O'Hara, K., & Dixon, D. (2011, May). Gamification. using game-design elements in non-gaming contexts. In CHI'11 extended abstracts on human factors in computing systems (pp. 2425–2428). ACM.

  32. Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-Herráiz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers and Education, 63, 380–392.

    Article  Google Scholar 

  33. Duke, D. M., Thirunarayanan, M., Byram, A., and Clarke, P. J. (2019). Student's perceptions of the implementation of a cyberlearning tool. In Proceedings of the 126th ASEE Annual Conference and Exposition - Computing and Information Technology Division, Paper No. 26250. Washington, DC: ASEE

  34. Farzan, R., DiMicco, J. M., Millen, D. R., Dugan, C., Geyer, W., & Brownholtz, E. A. (2008, April). Results from deploying a participation incentive mechanism within the enterprise. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 563–572). ACM.

  35. Flatla, D.R., Gutwin, C., Nacke, L.E., Bateman, S. and Mandryk, R.L. Calibration games: Making calibration tasks enjoyable by adding motivating game elements. In Proceedings UIST'11, ACM (2011).

  36. Foster, J. A., Sheridan, P. K., Irish, R., & Frost, G. S. (2012). Gamification as a strategy for promoting more in-depth investigation in a reverse engineering activity. In American Society for Engineering Education.

    Google Scholar 

  37. Frith, J. H. (2012). Constructing location, one check-in at a time: Examining the practices of Foursquare users.

  38. Fu, Y., & Clarke, P. (2016). Gamification based cyber enabled learning environment of software testing. In Submitted to the 123rd American Society for Engineering Education (ASEE)-Software Engineering Constituent.

  39. Gaffney, C., & Dunphy, N. Gender inclusivity dissemination guidelines (2015).

  40. Gåsland, M. (2011). Game mechanic based e-learning. Science and Technology, Master Thesis (June 2011). Retrieved October 4, 2014.

  41. Glover, I. (2013). Play as you learn: Gamification as a technique for motivating learners. In J. Herrington, et al. (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 1999–2008). Chesapeake, VA: AACE.

  42. Gnauk, B., Dannecker, L., & Hahmann, M. (2012, March). Leveraging gamification in demand dispatch systems. In Proceedings of the 2012 Joint EDBT/ICDT Workshops (pp. 103–110). ACM.

  43. Godwin, A., Potvin, G., Hazari, Z., & Lock, R. (2016). Identity, critical agency, and engineering: An affective model for predicting engineering as a career choice. Journal of Engineering Education, 105(2), 312–340.

    Article  Google Scholar 

  44. Goehle, G. (2013). Gamification and web-based homework. Primus, 23(3), 234–246.

    Article  Google Scholar 

  45. Hakulinen, L., Auvinen, T., & Korhonen, A. (2015). The effect of achievement badges on students’ behavior: An empirical study in a university-level computer science course. International Journal of Emerging Technologies in Learning (iJET), 10(1), 18–29.

    Article  Google Scholar 

  46. Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work?—A literature review of empirical studies on gamification. In 2014 47th Hawaii International Conference on System Sciences, IEEE (pp. 3025–3034).

  47. Hazari, Z., Sonnert, G., Sadler, P. M., & Shanahan, M. C. (2010). Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study. Journal of research in science teaching, 47(8), 978–1003.

    Google Scholar 

  48. He, J., & Freeman, L. (2010). Are men more technology-oriented than women? The role of gender on the development of general computer self-efficacy of college students. Journal of Information Systems Education, 21(2), 203–213.

    Google Scholar 

  49. Hisham, F. B. M. N., Sulaiman, S. (2017). Adapting gamification approach in massive open online courses to improve user engagement. In UTM Computing Proceedings Innovation in Computing Technology and Applications (vol. 2, pp. 1–6). Skudai, Malaysia: UTM

  50. Huang, B., & Hew, K. F. (2015, November). Do points, badges and leaderboard increase learning and activity: A quasi-experiment on the effects of gamification. In Proceedings of the 23rd International Conference on Computers in Education (pp. 275–280).

  51. Kapp, K. M. (2012). The gamification of learning and instruction (p. 93). San Francisco: Wiley.

    Google Scholar 

  52. Kargarmoakhar, M. M. (2019, June). How cultural messages through experiences influence occupational pursuit of Muslim female computer science students. In 126th Annual Conference and Exposition of American Society for Engineering Education.

  53. Kargarmoakhar, M., Lunn, S., Zahedi, L., Ross, M., Hazari, Z., Weiss, M. A., Georgiopoulos, M., Christensen, K., & Solis, T. (2020, October). Understanding the experiences that contribute to the inclusion of underrepresented groups in computing. In 2020 IEEE Frontiers in Education Conference (FIE), IEEE (pp. 1–9).

  54. Khan, A., Ahmad, F. H., & Malik, M. M. (2017). Use of digital game based learning and gamification in secondary school science: The effect on student engagement, learning and gender difference. Education and Information Technologies, 22(6), 2767–2804.

    Article  Google Scholar 

  55. Kim, B. (2015). Designing gamification in the right way. Library Technology Reports, 51(2), 29–35.

    Google Scholar 

  56. Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179–188.

    Article  Google Scholar 

  57. Li, W., Grossman, T., & Fitzmaurice, G. (2012, October). GamiCAD: A gamified tutorial system for first-time AutoCAD users. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (pp. 103–112). ACM.

  58. Liu, Y., Alexandrova, T., & Nakajima, T. (2011, December). Gamifying intelligent environments. In Proceedings of the 2011 International ACM Workshop on Ubiquitous Meta User Interfaces (pp. 7–12). ACM.

  59. Marczewski, A. (2013). Gamification: A simple introduction and a bit more, (self-published on Amazon Digital Services, 2013)In .Massung, E., Coyle, D., Cater, K. F., Jay, M., & Preist, C. . Kindle edition. (Loc, 1405). Using crowdsourcing to support pro-environmental community activism. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2013, April, pp. 371–380). ACM.

  60. McDaniel, R., Lindgren, R., & Friskics, J. (2012, October). Using badges for shaping interactions in online learning environments. In Professional Communication Conference (IPCC), IEEE, 2012 (pp. 1–4). IEEE International

  61. McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. Penguin

  62. McLaren, B. M., Adams, D. M., Mayer, R. E., & Forlizzi, J. (2018). A computer-based game that promotes mathematics learning more than a conventional approach. In Gamification in Education: Breakthroughs in Research and Practice (pp. 415–437). IGI Global.

  63. Mirzaei, M., Sahebi, S., & Brusilovsky, P. (2019, June). Annotated examples and parameterized exercises: Analyzing students' behavior patterns. In International Conference on Artificial Intelligence in Education (pp. 308–319). Cham: Springer.

  64. Mirzaei, M., Sahebi, S., & Brusilovsky, P. (2020, May). Detecting trait versus performance student behavioral patterns using discriminative non-negative matrix factorization. In The Thirty-Third International Flairs Conference.

  65. Morschheuser, B., Hassan, L., Werder, K., & Hamari, J. (2018). How to design gamification? A method for engineering gamified software. Information and Software Technology, 95, 219–237.

    Article  Google Scholar 

  66. Newmann, F. M., Wehlage, G. G., & Lamborn, S. D. (1992). The significance and sources of student engagement. In F. M. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 11–39). Teachers College Press.

    Google Scholar 

  67. Orosz, G., Farkas, D., & Roland-Levy, C. P. (2013). Are competition and extrinsic motivation reliable predictors of academic cheating? Frontiers in Psychology, 4, 87.

    Article  Google Scholar 

  68. Ortiz-Rojas, M., Chiluiza, K., & Valcke, M. (2017). Gamification in computer programming: Effects on learning, engagement, self-efficacy and intrinsic motivation. In Proceedings of the European Conference on Games Based Learning, (pp. 507–514)

  69. Pedro, L. Z., Lopes, A. M., Prates, B. G., Vassileva, J., & Isotani, S. (2015, April). Does gamification work for boys and girls?: An exploratory study with a virtual learning environment. In Proceedings of the 30th Annual ACM Symposium on Applied Computing (pp. 214–219). ACM.

  70. Perez, T., Cromley, J. G., & Kaplan, A. (2014). The role of identity development, values, and costs in college STEM retention. Journal of Educational Psychology, 106(1), 315–329. https://doi.org/10.1037/a0034027

    Article  Google Scholar 

  71. Pierrakos, O., Beam, T. K., Constantz, J., Johri, A., & Anderson, R. (2009). On the development of a professional identity: Engineering persistors vs. engineering switchers. In Frontiers in Education Conference, 39(2), 599–604. http://doi.org/https://doi.org/10.1109/FIE.2009.5350571

  72. Pugh, K. J., Linnenbrink-Garcia, L., Koskey, K. L. K., Stewart, V. C., & Manzey, C. (2010). Motivation, learning, and transformative experience: A study of deep engagement in science. Science Education, 94(1), 1–28. https://doi.org/10.1002/sce.20344

    Article  Google Scholar 

  73. Reiners, T., & Wood, L. C. (2015). Gamification in education and business. Gamification in Education and Business, (October, pp. 1–710). doi: https://doi.org/10.1007/978-3-319-10208-5.

  74. Ross, M. (2016). A unicorn’s tale: Examining the experiences of Black women in engineering industry. West Lafayette: Purdue University.

    Google Scholar 

  75. Schreuders, Z. C., & Butterfield, E. M. (2016, August). Gamification for teaching and learning computer security in higher education. In 2016 USENIX Workshop on Advances in Security Education (ASE 16). USENIX Association.

  76. Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of human-computer studies, 74, 14–31.

    Article  Google Scholar 

  77. Smith, Rachel S. (2004). Guidelines for authors of learning objects. The New Media Consortium, 2004. Retrieved from https://files.eric.ed.gov/fulltext/ED505110.pdf (Accessed November 2019)

  78. Snyder, E., & Hartig, J. R. (2013). Gamification of board review: A residency curricular innovation. Medical education, 47(5), 524–525.

    Article  Google Scholar 

  79. Stott, A., & Neustaedter, C. (2013). Analysis of gamification in education. Surrey, BC, Canada, 8, 36.

    Google Scholar 

  80. Stryker, S., & Burke, P. J. (2000). The past, present, and future of an identity theory. Social psychology quarterly, 1, 284–297.

    Article  Google Scholar 

  81. Team, R. C. (2013). R: A language and environment for statistical computing.

  82. Taheri, M., Ross, M., Hazari, Z., Weiss, M., Georgiopoulos, M., Christensen, K., Solis, T., Garcia, A., & Chari, D. (2018, October). A structural equation model analysis of computing identity sub-constructs and student academic persistence. In 2018 IEEE Frontiers in Education Conference (FIE), IEEE (pp. 1–7)

  83. Thom, J., Millen, D., and DiMicco, J. Removing gamification from an enterprise SNS. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, ACM (2012), (pp. 1067–1070).

  84. Tsay, C. H. H., Kofinas, A., & Luo, J. (2018). Enhancing student learning experience with technology-mediated gamification: An empirical study. Computers and Education, 121, 1–17.

    Article  Google Scholar 

  85. Venkatesh, V., & Morris, M. G. (2000). Why don’t men ever stop to ask for directions? Gender, social influence, and their role in technology acceptance and usage behavior. MIS Quarterly, 24(1), 115–139.

    Article  Google Scholar 

  86. What is gamification. (2020). Retrieved May 2020, from http://engagementalliance.org/what-is-gamification/

  87. Wilson, B. C. (2002). A study of factors promoting success in computer science including gender differences. Computer Science Education, 12(1–2), 141–164.

    Article  Google Scholar 

  88. Witt, M., Scheiner, C. W., & Robra-Bissantz, S. (2011, October). Gamification of online idea competitions: Insights from an explorative case. In GI-Jahrestagung (p. 392).

  89. Zahedi, M. L. (2019, June). Implications of gamification in learning environments on computer science students: A comprehensive study. In 126th Annual Conference and Exposition of American Society for Engineering Education.

  90. Zahedi, L., & Lunn, S. J., & Pouyanfar, S., & Ross, M. S., & Ohland, M. W. (2020, June), Leveraging Machine-learning Techniques to Analyze Computing Persistence in Undergraduate Programs In Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual Online: https://doi.org/10.18260/1-2--34921

  91. Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. “O'Reilly Media, Inc.”

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. (NSF 1712116). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We would also like to thank all of our participants—faculty and students.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Leila Zahedi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zahedi, L., Batten, J., Ross, M. et al. Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development. J Comput High Educ (2021). https://doi.org/10.1007/s12528-021-09271-5

Download citation

Keywords

  • Gamification
  • Computer science
  • Gender differences
  • Education