Abstract
Recommending students useful and effective learning paths is highly valuable to improve their learning experience. The evaluation of the effectiveness of this recommendation is a challenging task that can be performed online or offline. Online evaluation is highly popular but it relies on actual path recommendations to students, which may have dramatic implications. Offline evaluation relies on static datasets of students’ learning activities and simulates paths recommendations. Although easier to run, it is difficult to accurately evaluate offline the effectiveness of a learning path recommendation. To tackle this issue, this work proposes simple offline evaluation measures. We show that they actually allow to characterise and differentiate the algorithms.
Keywords
- Learning path recommendation
- Offline evaluation
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Durand, G., Laplante, F., Kop, R.: A learning design recommendation system based on Markov decision processes. In: 17th ACM KDD (2011)
Durand, G., Belacel, N., LaPlante, F.: Graph theory based model for learning path recommendation. Inf. Sci. 251, 10–21 (2013)
Dwivedi, P., Kant, V., Bharadwaj, K.: Learning path recommendation based on modified variable length genetic algorithm. Educ. Inf. Technol. 23(2), 819–836 (2018)
Embretson, S.E., Reise, S.P.: Item Response Theory. Psychology Press, New York (2013)
Feng, S., Li, X., Zeng, Y., Cong, G., Chee, Y.M., Yuan, Q.: Personalized ranking metric embedding for next new POI recommendation. In: 24th IJCAI (2015)
Hsieh, T., Wang, T.: A mining-based approach on discovering courses pattern for constructing suitable learning path. Expert Syst. Appl. 37(6), 4156–4167 (2010)
Léonard, M., Peter, Y., Secq, Y.: Patterns and loops: early computational thinking. In: Scheffel, M., Broisin, J., Pammer-Schindler, V., Ioannou, A., Schneider, J. (eds.) EC-TEL 2019. LNCS, vol. 11722, pp. 280–293. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29736-7_21
Liu, Q., et al.: Exploiting cognitive structure for adaptive learning. In: Proceedings of the 25th KDD, pp. 627–635 (2019)
Monti, D., Palumbo, E., Rizzo, G., Morisio, M.: Sequeval: an offline evaluation framework for sequence-based RS. Information 10(5), 174 (2019)
Nabizadeh, A., Gonçalves, D., Gama, S., Jorge, J., Rafsanjani, H.: Adaptive learning path recommender approach using auxiliary learning objects. Comput. Educ. 147 (2020)
Nabizadeh, A., Jorge, A., Leal, J.: Estimating time and score uncertainty in generating successful learning paths under time constraints. Expert Syst. 36(2), e12351 (2019)
Nabizadeh, A., Jorge, A., Leal, J.P.: Long term goal oriented recommender systems. In: Proceedings of the 11th WEBIST, pp. 552–557 (2015)
Quadrana, M., Cremonesi, P., Jannach, D.: Sequence-aware recommender systems. ACM Comput. Surv. (CSUR) 51(4), 1–36 (2018)
Rossetti, M., Stella, F., Zanker, M.: Contrasting offline and online results when evaluating recommendation algorithms. In: Proceedings of the 10th RecSys, pp. 31–34 (2016)
Su, C.: Designing and developing a novel hybrid adaptive learning path recommendation system (ALPRS) for gamification mathematics geometry course. Eurasia J. Math. Sci. Technol. Educ. 13(6), 2275–2298 (2017)
Su, J., Tseng, S., Wang, W., Weng, J., Yang, J., Tsai, W.: Learning portfolio analysis and mining for SCORM compliant environment. J. Educ. Technol. Soc. 9(1), 262–275 (2006)
Taraghi, B., Saranti, A., Ebner, M., Schön, M.: Markov chain and classification of difficulty levels enhances the learning path in one digit multiplication. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2014. LNCS, vol. 8523, pp. 322–333. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07482-5_31
Venant, R., Sharma, K., Vidal, P., Dillenbourg, P., Broisin, J.: Using sequential pattern mining to explore learners’ behaviors and evaluate their correlation with performance in inquiry-based learning. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 286–299. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_21
Vesin, B., Klašnja-Milićević, A., Ivanović, M., Budimac, Z.: Applying recommender systems and adaptive hypermedia for e-learning personalization. Comput. Inform. 32(3), 629–659 (2013)
Xie, H., Zou, D., Wang, F.L., Wong, T.L., Rao, Y., Wang, S.H.: Discover learning path for group users: a profile-based approach. Neurocomputing 254, 59–70 (2017)
Zhou, Y., Huang, C., Hu, Q., Zhu, J., Tang, Y.: Personalized learning full-path recommendation model based on LSTM neural networks. Inf. Sci. 444, 135–152 (2018)
Zhu, H., et al.: A multi-constraint learning path recommendation algorithm based on knowledge map. Knowl.-Based Syst. 143, 102–114 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Z., Brun, A., Boyer, A. (2020). New Measures for Offline Evaluation of Learning Path Recommenders. In: Alario-Hoyos, C., Rodríguez-Triana, M.J., Scheffel, M., Arnedillo-Sánchez, I., Dennerlein, S.M. (eds) Addressing Global Challenges and Quality Education. EC-TEL 2020. Lecture Notes in Computer Science(), vol 12315. Springer, Cham. https://doi.org/10.1007/978-3-030-57717-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-57717-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57716-2
Online ISBN: 978-3-030-57717-9
eBook Packages: Computer ScienceComputer Science (R0)