Advertisement

Process mining for self-regulated learning assessment in e-learning

  • Rebeca CerezoEmail author
  • Alejandro Bogarín
  • María Esteban
  • Cristóbal Romero
Article

Abstract

Content assessment has broadly improved in e-learning scenarios in recent decades. However, the e-Learning process can give rise to a spatial and temporal gap that poses interesting challenges for assessment of not only content, but also students’ acquisition of core skills such as self-regulated learning. Our objective was to discover students’ self-regulated learning processes during an e-Learning course by using Process Mining Techniques. We applied a new algorithm in the educational domain called Inductive Miner over the interaction traces from 101 university students in a course given over one semester on the Moodle 2.0 platform. Data was extracted from the platform’s event logs with 21,629 traces in order to discover students’ self-regulation models that contribute to improving the instructional process. The Inductive Miner algorithm discovered optimal models in terms of fitness for both Pass and Fail students in this dataset, as well as models at a certain level of granularity that can be interpreted in educational terms, which are the most important achievement in model discovery. We can conclude that although students who passed did not follow the instructors’ suggestions exactly, they did follow the logic of a successful self-regulated learning process as opposed to their failing classmates. The Process Mining models also allow us to examine which specific actions the students performed, and it was particularly interesting to see a high presence of actions related to forum-supported collaborative learning in the Pass group and an absence of those in the Fail group.

Keywords

e-Learning Self-regulated learning Educational process mining Educational data mining Inductive miner 

Notes

Acknowledgments

This work was funded by the Department of Science and Innovation (Spain) under the National Program for Research, Development and Innovation: project TIN2017-83445-P. We have also received funds from the European Union, through the European Regional Development Funds (ERDF); and the Principality of Asturias, through its Science, Technology and Innovation Plan FC-GRUPIN-IDI/2018/000199.

References

  1. Aljawarneh, S., Muhsin, Z., Nsour, A., Alkhateeb, F., & AlMaghayreh, E. (2010). E-learning tools and technologies in education: A perspective. In The fifth international conference of learning international networks consortium (LINC). Cambridge, MA: MIT. Retrieved from http://people.math.sfu.ca/~vjungic/shadi.pdf. Accessed 5 Jan 2019.
  2. Azevedo, R., & Aleven, V. (Eds.). (2013). International handbook of metacognition and learning technologies. Amsterdam: Springer.Google Scholar
  3. Azevedo, R., & Feyzi-Behnagh, R. (2011). Dysregulated learning with advanced learning technologies. Journal of e-Learning and Knowledge Society, 7(2), e9–e18.Google Scholar
  4. Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., et al. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). New York: Springer.CrossRefGoogle Scholar
  5. Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9(2), 161–185.  https://doi.org/10.1007/s11409-013-9107-6.CrossRefGoogle Scholar
  6. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379–439.CrossRefGoogle Scholar
  7. Biggs, J. B. (2005). Calidad del aprendizaje universitario (Quality of university learning). Madrid: Narcea.Google Scholar
  8. Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444.  https://doi.org/10.1146/annurev-psych-113011-143823.CrossRefGoogle Scholar
  9. Bogarín, A., Cerezo, R., & Romero, C. (2018a). A survey on educational process mining. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery.  https://doi.org/10.1002/widm.1230.Google Scholar
  10. Bogarin, A., Cerezo, R., Romero, C. (2018b). Discovering learning processes using inductive miner: A case study with learning management systems (LMSs). Psicothema, 30(3), 322–329.  https://doi.org/10.7334/psicothema2018.116.Google Scholar
  11. Bogarín, A., Romero, C., Cerezo, R., & Sánchez-Santillán, M. (2014). Clustering for improving educational process mining. In M. Pistilli, J. Willis, & D. Koch (Eds.), Proceedings of the fourth international conference on learning analytics and knowledge (pp. 170–181). Indianapolis: ACM.  https://doi.org/10.1145/2567574.2567604.Google Scholar
  12. Buijs, J. C., Van Dongen, B. F., & van Der Aalst, W. M. (2012). On the role of fitness, precision, generalization and simplicity in process discovery. In R. Meersman, H. Panetto, T. Dillon, S. Rinderle-Ma, P. Dadam, X. Zhou, S. Pearson, A. Ferscha, S. Bergamaschi, & I. F. Cruz (Eds.), Proceedings of the OTM confederated international conferences “on the move to meaningful internet systems” (pp. 305–322). Berlin: Springer.  https://doi.org/10.1007/978-3-642-33606-5_19.Google Scholar
  13. Cerezo, R., Esteban, M., Sánchez-Santillán, M., & Núñez, J. C. (2017). Procrastinating behavior in computer-based learning environments to predict performance: A case study in Moodle. Frontiers in Psychology, 8, 1403.CrossRefGoogle Scholar
  14. Cerezo, R., Fernández, E., Amieiro, N., Valle, A., Rosário, P., & Núñez, J. C. (2018). Mediating role of self-efficacy and usefulness between self-regulated learning strategy knowledge and its use. Revista de Psicodidáctica.  https://doi.org/10.1016/j.psicod.2018.08.001.Google Scholar
  15. Cerezo, R., Nuñez, J. C., Rosario, P., Valle, A., Rodriguez, S., & Bernardo, A. (2010). New Media for the promotion of self-regulated learning in higher education. Psicothema, 22(2), 306–315.Google Scholar
  16. Cerezo, R., Romero, C., Bogarín, A., & Núñez, J.C. (2014). Improving performance and comprehensibility of educational process mining models for a better understanding of the learning process. In Metacognition 2014. 6th Bienal meeting of the EARLI Special Interest Group 16. Estambul, Turquia (pp. 1–2).Google Scholar
  17. Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96, 42–54.  https://doi.org/10.1016/j.compedu.2016.02.006.CrossRefGoogle Scholar
  18. Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. New Jersey: Wiley.CrossRefGoogle Scholar
  19. Commission, European. (2014). New modes of learning and teaching in higher education. Luxembourg: European Union.Google Scholar
  20. Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. JAMA, 300(10), 1181–1196.CrossRefGoogle Scholar
  21. Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The current ecosystem of learning management systems in higher education: Student, faculty, and IT perspectives (Research report). Retrieved from http://www.educause.edu/ecar.2014EDUCAUSE.CCby-nc-nd. Accessed 20 Dec 2018.
  22. Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 9–17). ACM.  https://doi.org/10.1145/2090116.2090118.
  23. Emond, B., & Buffett, S. (2015). Analyzing student inquiry data using process discovery and sequence classification. Paper presented at the International Educational Data Mining Society, Madrid, Spain.Google Scholar
  24. Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). The KDD process for extracting useful knowledge from volumes of data. Communications of the ACM, 39(11), 27–34.  https://doi.org/10.1145/240455.240464.CrossRefGoogle Scholar
  25. García-Peñalvo, F. J., & Seoane Pardo, A. M. (2015). Una revisión actualizada del concepto de eLearning. Education in the Knowledge Society, 16, 119–144.  https://doi.org/10.14201/eks2015161119144.CrossRefGoogle Scholar
  26. Ge, X., & Harde, P. L. (2010). Self-processes and learning environment as influences in the development of expertise in instructional design. Learning Environment Research, 13(1), e23–e41.CrossRefGoogle Scholar
  27. Gómez-Aguilar, D. A., Hernández-García, A., García-Penñalvo, F. J., & Theron, R. (2015). Tap into visual analysis of customization of grouping of activities in eLearning. Computers in Human Behavior, 47, e60–e67.CrossRefGoogle Scholar
  28. Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance, 30, 65–84.Google Scholar
  29. Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Oxon: Routledge.CrossRefGoogle Scholar
  30. Hu, Y. H., Lo, C. L., & Shih, S. P. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behavior, 36, 469–478.  https://doi.org/10.1016/j.chb.2014.04.002.CrossRefGoogle Scholar
  31. Lara, J. A., Lizcano, D., Martínez, M. A., Pazos, J., & Riera, T. (2014). A system for knowledge discovery in e-learning environments within the European Higher Education Area-Application to student data from Open University of Madrid, UDIMA. Computers & Education, 72, 23–36.CrossRefGoogle Scholar
  32. Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2013). Discovering block-structured process models from event logs containing infrequent behaviour. Paper Presented at the International Conference on Business Process Management, Beijing, China.Google Scholar
  33. Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2014). Process and deviation exploration with inductive visual miner. BPM (Demos), 1295, 46.Google Scholar
  34. Lehmann, T., Hähnlein, I., & Ifenthaler, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior, 32, e313–e323.  https://doi.org/10.1016/j.chb.2013.07.051.CrossRefGoogle Scholar
  35. Lust, G., Elen, J., & Clarebout, G. (2013). Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education, 60(1), 385–395.CrossRefGoogle Scholar
  36. Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Jessup: US Department of Education.Google Scholar
  37. Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40.  https://doi.org/10.1016/j.compedu.2013.07.033.CrossRefGoogle Scholar
  38. Núñez, J. C., Cerezo, R., Bernardo, A., Rosário, P., Valle, A., Fernández, E., et al. (2011). Implementation of training programs in self-regulated learning strategies in moodle format: Results of a experience in higher education. Psicothema, 23(2), e274–e281.Google Scholar
  39. Paechter, M., Maier, B., & Macher, D. (2010). Evaluation universitärer Lehre mittels Einschätzungen des subjektiven Kompetenzerwerbs. Psychologie in Erziehung und Unterricht, 2, 128–138.CrossRefGoogle Scholar
  40. Reimann, P., Markauskaite, L., & Bannert, M. (2014). E-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540.  https://doi.org/10.1111/bjet.12146.CrossRefGoogle Scholar
  41. Romero, C., Cerezo, R., Bogarín, A., & Sánchez-Santillán, M. (2016). Educational process mining: a tutorial and case study using Moodle data sets. In Data mining and learning analytics: Applications in educational research (pp. 1–28). Wiley & Blackwell.  https://doi.org/10.1002/9781118998205.ch1.
  42. Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers & Education, 68, 458–472.CrossRefGoogle Scholar
  43. Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27.Google Scholar
  44. Romero, C., Ventura, S., & y García, A. E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384.CrossRefGoogle Scholar
  45. Sanchez-Santillan, M., Paule-Ruiz, M., Cerezo, R., & Alvarez-García, V. (2016). MeL: Modelo de adaptación dinámica del proceso de aprendizaje en eLearning. Anales de psicología, 32(1), 106–114.CrossRefGoogle Scholar
  46. Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R., Abrami, P. C., Wade, C. A., et al. (2009). Technology’s effect on achievement in higher education: A Stage I meta-analysis of classroom applications. Journal of Computing in Higher Education, 21(2), 95–109.CrossRefGoogle Scholar
  47. van der Aalst, W. M. (2011). Process discovery: An introduction. In W. M. P. van der Aalst, (Ed.), Process mining (pp. 125–156). Berlin, Heidelberg: Springer.  https://doi.org/10.1007/978-3-642-19345-3_5.CrossRefGoogle Scholar
  48. van der Aalst, W. M. (2016). Process mining: Data science in action. Berlin: Springer.  https://doi.org/10.1007/978-3-662-49851-4.CrossRefGoogle Scholar
  49. Van der Aalst, W., Adriansyah, A., & van Dongen, B. (2012). Replaying history on process models for conformance checking and performance analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2(2), 182–192.Google Scholar
  50. Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J., & Hlosta, M. (2014). Developing predictive models for early detection of at-risk students on distance learning modules. In Machine learning and learning analytics workshop at the 4th international conference on learning analytics and knowledge (LAK14), 24–28 March 2014, Indianapolis, IN, USA. Retrieved from http://lak14indy.wordpress.com/. Accessed 5 Jan 2019.
  51. Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational psychologist, 25(1), 3–17.CrossRefGoogle Scholar
  52. Zimmerman, B. J. (2013). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), self-regulated learning and academic achievement (pp. 10–45). London: Routledge.Google Scholar
  53. Zimmerman, B. J., & Schunk, D. (2011). Handbook of self-regulation of learning and performance. New York: Routledge.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of PsychologyUniversity of OviedoOviedoSpain
  2. 2.University of CórdobaCórdobaSpain
  3. 3.University of OviedoOviedoSpain

Personalised recommendations