Advertisement

Universal Access in the Information Society

, Volume 18, Issue 3, pp 533–549 | Cite as

Understanding student behavior and perceptions toward earning badges in a gamified MOOC

  • Alejandro Ortega-ArranzEmail author
  • Erkan Er
  • Alejandra Martínez-Monés
  • Miguel L. Bote-Lorenzo
  • Juan I. Asensio-Pérez
  • Juan A. Muñoz-Cristóbal
Long Paper

Abstract

Despite the advantages of MOOCs, such as the open and free access to education, these courses are criticized for students’ lack of motivation and their high dropout rates. Gamification is a technique used to increase student motivation and engagement in small-scale educational contexts. However, the effects of gamification on student engagement have been scarcely explored in MOOC environments, and the findings so far are inconsistent. To address this gap, this research work examines the students’ behavior toward earning badges and how it relates to their engagement in a gamified MOOC. According to the results, the behaviors toward badges of the active students were generally positive and significantly correlated with other variables measuring their engagement (e.g., pageviews, submitted tasks, forum posts), although this positive behavior seems to decrease throughout the course. Additionally, students that reported high motivation by badges at the end of the course showed a higher engagement level than those that were not appealed by badges.

Keywords

Gamification MOOC Engagement Perceptions Badges Rewards 

Notes

Acknowledgements

This research has been partially funded by the Spanish State Research Agency (AEI) together with the European Regional Development Fund, under Project Grants TIN2014-53199-C3-2-R and TIN2017-85179-C3-2-R, the Regional Government of Castilla y León together with the European Regional Development Fund, under Project Grants VA082U16 and VA257P18, and the European Commission, under Project Grant 588438-EPP-1-2017-1-EL-EPPKA2-KA. The authors would like to thank the rest of the GSIC-EMIC research team for their valuable ideas and the Canvas Network team for the support received conducting this research.

References

  1. 1.
    Alario-Hoyos, C., Pérez-Sanagustín, M., Cormier, D., Delgado Kloos, C.: Proposal for a conceptual framework for educators to describe and design MOOCs. J. UCS 20(1), 6–23 (2014)Google Scholar
  2. 2.
    Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado Kloos, C., Parada G, H.A., Muñoz-Organero, M.: Delving into participants’ profiles and use of social tools in MOOCs. IEEE Trans. Learn. Technol. 3(7), 260–266 (2014)CrossRefGoogle Scholar
  3. 3.
    Anderson, A., Huttenlocher, D., Kleinberg, J., Leskovec, J.: Engaging with massive online courses. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 687–698. ACM (2014)Google Scholar
  4. 4.
    Antonaci, A., Klemke, R., Stracke, C.M., Specht, M.: Gamification in MOOCs to enhance users’ goal achievement. In: Proceedings of the 2017 Global Engineering Education Conference, pp. 1654–1662. IEEE (2017)Google Scholar
  5. 5.
    Aparicio, M., Oliveira, T., Bacao, F., Painho, M.: Gamification: a key determinant of massive open online course (MOOC) success. Inf. Manag. 56(1), 39–54 (2019)CrossRefGoogle Scholar
  6. 6.
    Bartle, R.: Hearts, clubs, diamonds, spades: players who suit MUDs. J. MUD Res. 1(1), 19 (1996)Google Scholar
  7. 7.
    Bethlehem, J.: Selection bias in web surveys. Int. Stat. Rev. 78(2), 161–188 (2010)CrossRefGoogle Scholar
  8. 8.
    Cross, S., Whitelock, D., Galley, R.: The use, role and reception of open badges as a method for formative and summative reward in two massive open online courses. Int. J. e-Assess. 4(1), 1–16 (2014)Google Scholar
  9. 9.
    Csikszentmihalyi, M.: Beyond Boredom and Anxiety. Jossey-Bass, San Francisco (2000)Google Scholar
  10. 10.
    Daniel, J.: Making sense of MOOCs: musings in a maze of myth, paradox and possibility. J. Interact. Media Educ. 2012(3), 18 (2012)CrossRefGoogle Scholar
  11. 11.
    De Sousa Borges, S., Durelli, V., Reis, H., Isotani, S.: A systematic mapping on gamification applied to education. In: Proceedings of the 29th Annual ACM Symposium on Applied Computing, pp. 216–222. ACM (2014)Google Scholar
  12. 12.
    Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining gamification. In: Proceedings of the 15th ACM International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9–15 (2011)Google Scholar
  13. 13.
    Dicheva, D., Dichev, C., Agre, G., Angelova, G.: Gamification in education: a systematic mapping study. J. Educ. Technol. Soc. 18(3), 75–88 (2015)Google Scholar
  14. 14.
    Dillenbourg, P., Fox, A., Kirchner, C., Mitchell, J., Wirsing, M.: Massive open online courses: current state and perspectives (Dagstuhl perspectives workshop 14112). Dagstuhl Manif. 4, 1–27 (2014)Google Scholar
  15. 15.
    Ding, L., Kim, C., Orey, M.: Studies of student engagement in gamified online discussions. Comput. Educ. 115, 126–142 (2017)CrossRefGoogle Scholar
  16. 16.
    Domínguez, A., Saenz-de Navarrete, J., De-Marcos, L., Fernández-Sanz, L., Pagés, C., Martínez-Herráiz, J.J.: Gamifying learning experiences: practical implications and outcomes. Comput. Educ. 63, 380–392 (2013)CrossRefGoogle Scholar
  17. 17.
    Ferguson, R., Sharples, M.: Innovative pedagogy at massive scale: teaching and learning in MOOCs. In: Open Learning and Teaching in Educational Communities, pp. 98–111. Springer, Berlin (2014)Google Scholar
  18. 18.
    Festinger, L.: A theory of social comparison processes. Hum. Relat. 7(2), 117–140 (1954)CrossRefGoogle Scholar
  19. 19.
    Fitz-Walter, Z.: Achievement Unlocked: Investigating the Design of Effective Gamification Experiences for Mobile Applications and Devices. Ph.D. Thesis, Queensland University of Technology, Australia (2015)Google Scholar
  20. 20.
    Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)CrossRefGoogle Scholar
  21. 21.
    Fresno, J., Ortega-Arranz, H., Ortega-Arranz, A., González-Escribano, A., Llanos, D.R.: Applying Gamification in a Parallel Programming Course. Gamification-Based E-Learning Strategies for Computer Programming Education. IGI Global, Hershey (2016)Google Scholar
  22. 22.
    Greene, J.C.: Mixed Methods in Social Inquiry, vol. 9. Wiley, Hoboken (2007)Google Scholar
  23. 23.
    Hakulinen, L., Auvinen, T., Korhonen, A.: Empirical study on the effect of achievement badges in TRAKLA2 online learning environment. In: Proceedings of the 2013 International Conference on Learning and Teaching in Computing and Engineering, pp. 47–54. IEEE (2013)Google Scholar
  24. 24.
    Hamari, J.: Transforming homo economicus into homo ludens: a field experiment on gamification in a utilitarian peer-to-peer trading service. Electron. Commer. Res. Appl. 12(4), 236–245 (2013)CrossRefGoogle Scholar
  25. 25.
    Hamari, J.: Do badges increase user activity? A field experiment on the effects of gamification. Comput. Hum. behav. 71, 469–478 (2017)CrossRefGoogle Scholar
  26. 26.
    Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work? A literature review of empirical studies on gamification. In: Proceedings of the 47th Hawaii International Conference on System Sciences, pp. 3025–3034. IEEE (2014)Google Scholar
  27. 27.
    Henderikx, M.A., Kreijns, K., Kalz, M.: Refining success and dropout in massive open online courses based on the intention-behavior gap. Distance Educ. 38(3), 353–368 (2017)CrossRefGoogle Scholar
  28. 28.
    Henrie, C.R., Halverson, L.R., Graham, C.R.: Measuring student engagement in technology-mediated learning: a review. Comput. Educ. 90, 36–53 (2015)CrossRefGoogle Scholar
  29. 29.
    Hew, K.F.: Promoting engagement in online courses: what strategies can we learn from three highly rated MOOCS. Br. J. Educ. Technol. 47(2), 320–341 (2016)CrossRefGoogle Scholar
  30. 30.
    Ibáñez, M.B., Di-Serio, A., Delgado Kloos, C.: Gamification for engaging computer science students in learning activities: a case study. IEEE Trans. Learn. Technol. 7(3), 291–301 (2014)CrossRefGoogle Scholar
  31. 31.
    Jordan, K.: Initial trends in enrolment and completion of massive open online courses. Int. Rev. Res. Open Distrib. Learn. 15(1), 133–160 (2014)Google Scholar
  32. 32.
    Khalil, H., Ebner, M.: MOOCs completion rates and possible methods to improve retention—a literature review. In: Proceedings of the 2014 World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1305–1313 (2014)Google Scholar
  33. 33.
    Khalil, M., Ebner, M.: Learning analytics in MOOCs: can data improve students retention and learning? In: Proceedings of the 2016 World Conference on Educational Media and Technology, pp. 581–588. Association for the Advancement of Computing in Education (AACE) (2016)Google Scholar
  34. 34.
    Khalil, M., Ebner, M., Admiraal, W.: How can gamification improve MOOC student engagement? In: Proceedings of the 11th European Conference on Games Based Learning, pp. 819–828 (2017)Google Scholar
  35. 35.
    Khalil, M., Wong, J., de Koning, B., Ebner, M., Paas, F.: Gamification in MOOCs: a review of the state of the art. In: Proceedings of the 2018 Global Engineering Education Conference, pp. 1629–1638. IEEE (2018)Google Scholar
  36. 36.
    Kyewski, E., Krämer, N.C.: To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Comput. Educ. 118, 25–37 (2018)CrossRefGoogle Scholar
  37. 37.
    Lee, J.J., Hammer, J.: Gamification in education: what, how, why bother? Acad. Exch. Q. 15(2), 146 (2011)Google Scholar
  38. 38.
    Liyanagunawardena, T.R., Lundqvist, K.Ø., Williams, S.A.: Who are with us: MOOC learners on a futurelearn course. Br. J. Educ. Technol. 46(3), 557–569 (2015)CrossRefGoogle Scholar
  39. 39.
    Mertler, C.A., Reinhart, R.V.: Advanced and Multivariate Statistical Methods: Practical Application and Interpretation. Routledge, Abingdon (2016)Google Scholar
  40. 40.
    Miles, M.B., Huberman, A.M.: Qualitative Data Analysis: An Expanded Sourcebook. SAGE Publications Inc., Thousand Oaks (1994)Google Scholar
  41. 41.
    Morales, M., Amado-Salvatierra, H.R., Hernández, R., Pirker, J., Gütl, C.: A practical experience on the use of gamification in MOOC courses as a strategy to increase motivation. In: Proceeding of the 2016 International Workshop on Learning Technology for Education in Cloud, pp. 139–149. Springer, Berlin (2016)Google Scholar
  42. 42.
    Mukaka, M.M.: A guide to appropriate use of correlation coefficient in medical research. Malawi Med. J. 24(3), 69–71 (2012)Google Scholar
  43. 43.
    Nakagawa, S., Cuthill, I.C.: Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 82(4), 591–605 (2007)CrossRefGoogle Scholar
  44. 44.
    Navidi, W.C.: Statistics for Engineers and Scientists. McGraw-Hill Higher Education, New York (2008)zbMATHGoogle Scholar
  45. 45.
    O’Donovan, S., Gain, J., Marais, P.: A case study in the gamification of a university-level games development course. In: Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference, pp. 242–251. ACM (2013)Google Scholar
  46. 46.
    Ortega-Arranz, A., Kalz, M., Martínez-Monés, A.: Creating engaging experiences in MOOC through in-course redeemable rewards. In: Proceedings of the 2018 Global Engineering Education Conference, pp. 1875–1882. IEEE (2018)Google Scholar
  47. 47.
    Ortega-Arranz, A., Muñoz-Cristóbal, J.A., Martínez-Monés, A., Bote-Lorenzo, M.L., Asensio-Pérez, J.I.: How gamification is being implemented in MOOCs? A systematic literature review. In: In Proceedings of the 12th European Conference on Technology Enhanced Learning, pp. 441–447. Springer, Berlin (2017)Google Scholar
  48. 48.
    Ortega-Arranz, A., Sanz-Martínez, L., Álvarez-Álvarez, S., Muñoz-Cristóbal, J.A., Bote-Lorenzo, M.L., Martínez-Monés, A., Dimitriadis, Y.: From low-scale to collaborative, gamified and massive-scale courses: redesigning a MOOC. In: Proceedings of the 5th European Conference on Massive Open Online Courses, pp. 77–87. Springer, Berlin (2017)Google Scholar
  49. 49.
    Reischer, M., Khalil, M., Ebner, M.: Does gamification in MOOC discussion forums work? In: Proceedings of the 5th European Conference on Massive Open Online Courses, pp. 95–101. Springer, Berlin (2017)Google Scholar
  50. 50.
    Rizzardini, R.H., Chan, M.M., Guetl, C.: Chapter 14 - An attrition model for MOOCs: evaluating the learning strategies of gamification. In: Caballé, S., Clarisó, R. (eds.) Formative Assessment, Learning Data Analytics and Gamification. Intelligent Data-Centric Systems, pp. 295–311. Academic Press, Boston (2016)Google Scholar
  51. 51.
    Ruipérez-Valiente, J.A., Muñoz-Merino, P.J., Delgado Kloos, C.: An analysis of the use of badges in an educational experiment. In: Frontiers in Education Conference (FIE), pp. 1–8. IEEE (2016)Google Scholar
  52. 52.
    Ruipérez-Valiente, J.A., Muñoz-Merino, P.J., Delgado Kloos, C.: Detecting and clustering students by their gamification behavior with badges: a case study in engineering education. Int. J. Eng. Educ. 33(2), 816–830 (2017)Google Scholar
  53. 53.
    Shah, D.: Monetization over Massiveness: A Review of MOOC Stats and Trends in 2016. Retrieved from: https://www.class-central.com/report/moocs-stats-and-trends-2016/ (2016). Accessed April 2017
  54. 54.
    Shah, D.: By the Numbers: MOOCS in 2017. How has the MOOC Space Grown This Year? Get the Facts, Figures, and Pie Charts. Retrieved from: https://www.class-central.com/report/mooc-stats-2017/ (2017). Accessed June 2018
  55. 55.
    Siemens, G.: Massive open online courses: innovation in education. Open Educ. Resour.: Innov., Res. Pract. 5, 5–15 (2013)Google Scholar
  56. 56.
    Stake, R.E.: The Art of Case Study Research. SAGE Publications Inc., Thousand Oaks (1995)Google Scholar
  57. 57.
    Staubitz, T., Willems, C., Hagedorn, C., Meinel, C.: The gamification of a MOOC platform. In: Proceedings of the 2017 Global Engineering Education Conference, pp. 883–892. IEEE (2017)Google Scholar
  58. 58.
    Werbach, K., Hunter, D.: For the Win: How Game Thinking Can Revolutionize Your Business. Wharton Digital Press, Philadelphia (2012)Google Scholar
  59. 59.
    Xu, X., Zhi, H.: Study of Game Elements Impacting on SE Course Completion Rate in MOOCs. Master’s Thesis, Faculty of Computing, Blekinge Institute of Technology, SE-371 79, Karlskrona (2017)Google Scholar
  60. 60.
    Zufferey, J.D., Alavi, H., Kidzinski, L., Dillenbourg, P.: Gamified competition features for corporate MOOCs: the battle mode. In: Proceedings of the 4th European MOOCs Stakeholder Summit, p. 223 (2016)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Telecommunications EngineeringUniversidad de ValladolidValladolidSpain
  2. 2.School of Computer EngineeringUniversidad de ValladolidValladolidSpain

Personalised recommendations