Process mining for self-regulated learning assessment in e-learning
Content assessment has broadly improved in e-learning scenarios in recent decades. However, the e-Learning process can give rise to a spatial and temporal gap that poses interesting challenges for assessment of not only content, but also students’ acquisition of core skills such as self-regulated learning. Our objective was to discover students’ self-regulated learning processes during an e-Learning course by using Process Mining Techniques. We applied a new algorithm in the educational domain called Inductive Miner over the interaction traces from 101 university students in a course given over one semester on the Moodle 2.0 platform. Data was extracted from the platform’s event logs with 21,629 traces in order to discover students’ self-regulation models that contribute to improving the instructional process. The Inductive Miner algorithm discovered optimal models in terms of fitness for both Pass and Fail students in this dataset, as well as models at a certain level of granularity that can be interpreted in educational terms, which are the most important achievement in model discovery. We can conclude that although students who passed did not follow the instructors’ suggestions exactly, they did follow the logic of a successful self-regulated learning process as opposed to their failing classmates. The Process Mining models also allow us to examine which specific actions the students performed, and it was particularly interesting to see a high presence of actions related to forum-supported collaborative learning in the Pass group and an absence of those in the Fail group.
Keywordse-Learning Self-regulated learning Educational process mining Educational data mining Inductive miner
This work was funded by the Department of Science and Innovation (Spain) under the National Program for Research, Development and Innovation: project TIN2017-83445-P. We have also received funds from the European Union, through the European Regional Development Funds (ERDF); and the Principality of Asturias, through its Science, Technology and Innovation Plan FC-GRUPIN-IDI/2018/000199.
- Aljawarneh, S., Muhsin, Z., Nsour, A., Alkhateeb, F., & AlMaghayreh, E. (2010). E-learning tools and technologies in education: A perspective. In The fifth international conference of learning international networks consortium (LINC). Cambridge, MA: MIT. Retrieved from http://people.math.sfu.ca/~vjungic/shadi.pdf. Accessed 5 Jan 2019.
- Azevedo, R., & Aleven, V. (Eds.). (2013). International handbook of metacognition and learning technologies. Amsterdam: Springer.Google Scholar
- Azevedo, R., & Feyzi-Behnagh, R. (2011). Dysregulated learning with advanced learning technologies. Journal of e-Learning and Knowledge Society, 7(2), e9–e18.Google Scholar
- Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., et al. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). New York: Springer.CrossRefGoogle Scholar
- Biggs, J. B. (2005). Calidad del aprendizaje universitario (Quality of university learning). Madrid: Narcea.Google Scholar
- Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823.CrossRefGoogle Scholar
- Bogarín, A., Romero, C., Cerezo, R., & Sánchez-Santillán, M. (2014). Clustering for improving educational process mining. In M. Pistilli, J. Willis, & D. Koch (Eds.), Proceedings of the fourth international conference on learning analytics and knowledge (pp. 170–181). Indianapolis: ACM. https://doi.org/10.1145/2567574.2567604.Google Scholar
- Buijs, J. C., Van Dongen, B. F., & van Der Aalst, W. M. (2012). On the role of fitness, precision, generalization and simplicity in process discovery. In R. Meersman, H. Panetto, T. Dillon, S. Rinderle-Ma, P. Dadam, X. Zhou, S. Pearson, A. Ferscha, S. Bergamaschi, & I. F. Cruz (Eds.), Proceedings of the OTM confederated international conferences “on the move to meaningful internet systems” (pp. 305–322). Berlin: Springer. https://doi.org/10.1007/978-3-642-33606-5_19.Google Scholar
- Cerezo, R., Nuñez, J. C., Rosario, P., Valle, A., Rodriguez, S., & Bernardo, A. (2010). New Media for the promotion of self-regulated learning in higher education. Psicothema, 22(2), 306–315.Google Scholar
- Cerezo, R., Romero, C., Bogarín, A., & Núñez, J.C. (2014). Improving performance and comprehensibility of educational process mining models for a better understanding of the learning process. In Metacognition 2014. 6th Bienal meeting of the EARLI Special Interest Group 16. Estambul, Turquia (pp. 1–2).Google Scholar
- Commission, European. (2014). New modes of learning and teaching in higher education. Luxembourg: European Union.Google Scholar
- Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The current ecosystem of learning management systems in higher education: Student, faculty, and IT perspectives (Research report). Retrieved from http://www.educause.edu/ecar.2014EDUCAUSE.CCby-nc-nd. Accessed 20 Dec 2018.
- Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 9–17). ACM. https://doi.org/10.1145/2090116.2090118.
- Emond, B., & Buffett, S. (2015). Analyzing student inquiry data using process discovery and sequence classification. Paper presented at the International Educational Data Mining Society, Madrid, Spain.Google Scholar
- Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance, 30, 65–84.Google Scholar
- Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2013). Discovering block-structured process models from event logs containing infrequent behaviour. Paper Presented at the International Conference on Business Process Management, Beijing, China.Google Scholar
- Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2014). Process and deviation exploration with inductive visual miner. BPM (Demos), 1295, 46.Google Scholar
- Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Jessup: US Department of Education.Google Scholar
- Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40. https://doi.org/10.1016/j.compedu.2013.07.033.CrossRefGoogle Scholar
- Núñez, J. C., Cerezo, R., Bernardo, A., Rosário, P., Valle, A., Fernández, E., et al. (2011). Implementation of training programs in self-regulated learning strategies in moodle format: Results of a experience in higher education. Psicothema, 23(2), e274–e281.Google Scholar
- Romero, C., Cerezo, R., Bogarín, A., & Sánchez-Santillán, M. (2016). Educational process mining: a tutorial and case study using Moodle data sets. In Data mining and learning analytics: Applications in educational research (pp. 1–28). Wiley & Blackwell. https://doi.org/10.1002/9781118998205.ch1.
- Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27.Google Scholar
- Van der Aalst, W., Adriansyah, A., & van Dongen, B. (2012). Replaying history on process models for conformance checking and performance analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2(2), 182–192.Google Scholar
- Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J., & Hlosta, M. (2014). Developing predictive models for early detection of at-risk students on distance learning modules. In Machine learning and learning analytics workshop at the 4th international conference on learning analytics and knowledge (LAK14), 24–28 March 2014, Indianapolis, IN, USA. Retrieved from http://lak14indy.wordpress.com/. Accessed 5 Jan 2019.
- Zimmerman, B. J. (2013). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), self-regulated learning and academic achievement (pp. 10–45). London: Routledge.Google Scholar
- Zimmerman, B. J., & Schunk, D. (2011). Handbook of self-regulation of learning and performance. New York: Routledge.Google Scholar