Using explanations for recommender systems in learning design settings to enhance teachers’ acceptance and perceived experience
- 48 Downloads
The reuse of Learning Designs can bring significant advantages to the educational community such as the diffusion of best teaching practices and the improvement of teaching quality and learning outcomes. Although various tools, including Recommender Systems, have been developed to implement the notion of reusing Learning Designs, their adoption by teachers falls short of expectations. This paper investigates the results of providing explanations for Learning Design recommendations to teachers. To this end, we designed and implemented an explanatory mechanism incorporated into a Recommender System, which propose pre-existing Learning Designs to teachers. We then conducted a user-centric evaluation experiment. Overall, this study provides evidence that explanations should be incorporated into Recommender Systems that propose Learning Designs, as a way of improving the teacher-perceived experience and promoting their wider adoption by teachers. The more teachers accept and adopt Recommender Systems that propose Learning Designs, the more the educational community gains the benefits of reusing Learning Designs.
KeywordsExplanations Recommender systems Learning design Technology enhanced learning
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- Agostinho, S. (2009). Learning Design Representations to Document, Model, and Share Teaching Practice. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of Research on Learning Design and Learning Objects: Issues, Applications, and Technologies (pp. 1–19). Hershey: IGI Global. https://doi.org/10.4018/978-1-59904-861-1.ch001.
- Agostinho, S., Bennett, S., Lockyer, L., Jones, J., & Harper, B. (2013). Learning designs as a stimulus and support for teachers’ design practices. In H. Beetham & R. Sharpe (Eds.), Rethinking Pedagogy for a Digital Age: Designing for 21st century (pp. 119–132). New York: Routledge.Google Scholar
- Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: designing and delivering e-learning. https://doi.org/10.4324/9780203961681.
- Bennett, S., Agostinho, S., & Lockyer, L. (2005). Reusable learning designs in university education. Proceedings of the IASTED International Conference on Education and Technology, ICET 2005, 2005, 102–106. Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-33751297745&partnerID=40&md5=de22754ab9711f446c2d71bc23695040. Accessed 27 Nov 2018.
- Bennett, S., Thomas, L., Agostinho, S., Lockyer, L., Jones, J., & Harper, B. (2011). Understanding the design context for Australian university teachers: implications for the future of learning design. Learning, Media and Technology, 36(2), 151–167. https://doi.org/10.1080/17439884.2011.553622.Google Scholar
- Cameron, L. (2009). How learning design can illuminate teaching practice. The Future of Learning Design Conference.Google Scholar
- Cameron, L., & Campbell, C. (2010). Sharing learning designs that work. In EdMedia: world Conference on Educational Media and Technology (pp. 1914–1919).Google Scholar
- Celik, D., & Magoulas, G. D. (2016). A review, timeline, and categorization of learning design tools. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 10013 LNCS, pp. 3–13). https://doi.org/10.1007/978-3-319-47440-3_1
- Chen, L., & Wang, F. (2017). Explaining recommendations based on feature sentiments in product reviews. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces (pp. 17–28). New York: ACM. https://doi.org/10.1145/3025171.3025173.
- Conole, G. (2013). Designing for learning in an open world. New York: Springer New York. Retrieved from https://doi.org/10.1007/978-1-4419-8517-0
- Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008). The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18(5), 455–496. https://doi.org/10.1007/s11257-008-9051-3.Google Scholar
- Dalziel, J. (2003). Implementing learning design : the learning activity management system (LAMS). Retrieved from http://hdl.handle.net/1959.14/79594. Accessed 28 Nov 2018.
- Fazeli, S., Drachsler, H., Bitter-Rijpkema, M., Brouns, F., van der Vegt, W., & Sloep, P. B. (2017). User-centric evaluation of recommender systems in social learning platforms: accuracy is just the tip of the iceberg. IEEE Transactions on Learning Technologies, 11, 294–306. https://doi.org/10.1109/TLT.2017.2732349.Google Scholar
- Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000). Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (pp. 241–250). New York: ACM. https://doi.org/10.1145/358916.358995
- Hernández-Leo, D., Harrer, A., Dodero, J. M., Asensio-Pérez, J. I., & Burgos, D. (2007). A framework for the conceptualization of approaches to create-by-reuse of learning design solutions. Journal of Universal Computer Science, 13(7), 750–760.Google Scholar
- Hernández-Leo, D., Chacón, J., Prieto, L. P., Asensio-Pérez, J. I., & Derntl, M. (2013). Towards an integrated learning design environment. In D. Hernández-Leo, T. Ley, R. Klamma, & A. Harrer (Eds.), Scaling up learning for sustained impact (pp. 448–453). Berlin: Springer Berlin Heidelberg.Google Scholar
- Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Prieto, L. P., & Chacón, J. (2014). ILDE: community environment for conceptualizing, authoring and deploying learning activities. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8719 LNCS, pp. 490–493). https://doi.org/10.1007/978-3-319-11200-8_48
- Karga, S., & Satratzemi, M. (2018). A hybrid recommender system integrated into LAMS for learning designers. Education and Information Technologies, 23(3), 1297–1329. https://doi.org/10.1007/s10639-017-9668-0.
- Koper, R., & Bennett, S. (2008). Learning design: Concepts. In H. Adelsberger, Kinshuk, J. Pawlowski, & D. Sampson (Eds.), (pp. 135–154). Springer Berlin Heidelberg. Retrieved from https://doi.org/10.1007/978-3-540-74155-8_8.
- Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., & Getoor, L. (2017). User preferences for hybrid explanations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 84–88). New York: ACM. https://doi.org/10.1145/3109859.3109915.
- Laurillard, D. (2012). Teaching as a design science. Building pedagogical patterns for learning and technology. Routledge: Taylor & Francis Group.Google Scholar
- Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E. A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15–30. https://doi.org/10.1111/j.1365-2729.2011.00458.x.Google Scholar
- Lockyer, L., Bennett, S., Agostinho, S., & Harper, B. (2009). Handbook of Research on Learning Design and Learning Objects. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.). IGI Global. https://doi.org/10.4018/978-1-59904-861-1.
- Maina, M., Craft, B., & Mor, Y. (2015). The Art & Science of Learning Design. Rotterdam: Sense Publishers. Retrieved from https://www.sensepublishers.com/catalogs/bookseries/technology-enhanced-learning-1/the-art-and-science-of-learning-design/. Accessed 27 Nov 2018.
- McNee, S., Lam, S., Konstan, J., & Riedl, J. (2003). Interfaces for Eliciting New User Preferences in Recommender Systems. User Modeling 2003, 2702, 148. https://doi.org/10.1007/3-540-44963-9_24.
- McNee, S. M., Riedl, J., & Konstan, J. a. (2006). Being accurate is not enough: How accuracy metrics have hurt recommender systems. CHI’06 Extended Abstracts on Human Factors in Computing Systems, 1101. https://doi.org/10.1145/1125451.1125659.
- Mor, Y., & Craft, B. (2012). Learning design: reflections on a snapshot of the current landscape. Research in Learning Technology, 20. Retrieved from http://www.researchinlearningtechnology.net/index.php/rlt/article/view/19196/. Accessed 27 Nov 2018.
- Muhammad, K. I., Lawlor, A., & Smyth, B. (2016). A live-user study of opinionated explanations for recommender systems. In Proceedings of the 21st International Conference on Intelligent User Interfaces (pp. 256–260). New York: ACM. https://doi.org/10.1145/2856767.2856813.
- Muñoz, O. R., Penalba, F. A., & Sánchez, J. F. (2016). The skills, competences, and attitude toward information and communications technology recommender system: an online support program for teachers with personalized recommendations. New Review of Hypermedia and Multimedia, 22(1–2), 83–110. https://doi.org/10.1080/13614568.2015.1036132.Google Scholar
- O’Donovan, J., Smyth, B., Gretarsson, B., Bostandjiev, S., & Höllerer, T. (2008). PeerChooser: visual interactive recommendation. In SIGCHI Conference on Human Factors in Computing Systems (pp. 1085–1088). Florence: ACM. https://doi.org/10.1145/1357054.1357222.
- Paquette, G., Teja, I. D. la, & K. Lundgren-Cayrol. (2006). Learning design classification definitions. Retrieved from http://helios.licef.ca/residld/4/Classification_Design_P%C3%A9dagogique.doc. Accessed 27 Nov 2018.
- Prieto, L., Dimitriadis, Y., Craft, B., Derntl, M., Émin, V., Katsamani, M., … Villasclaras, E. (2013). Learning design Rashomon II: Exploring one lesson through multiple tools. Research in Learning Technology, 21(0). https://doi.org/10.3402/rlt.v21i0.20057.
- Pu, P., & Chen, L. (2006). Trust building with explanation interfaces. In Proceedings of the 11th International Conference on Intelligent User Interfaces (pp. 93–100). New York: ACM. https://doi.org/10.1145/1111449.1111475.
- Pu, P., & Chen, L. (2011). A user - centric evaluation framework for recommender systems. Proceedings of the 5th ACM Conference on Recommender Systems - RecSys ‘11, 157–164. https://doi.org/10.1145/2043932.2043962.
- Quijano-Sanchez, L., Sauer, C., Recio-Garcia, J. A., & Diaz-Agudo, B. (2017). Make it personal: A social explanation system applied to group recommendations. Expert Systems with Applications, 76, 36–48. https://doi.org/10.1016/j.eswa.2017.01.045.
- Simon, B., Pulkkinen, M., Totschnig, M., & Kozlov, D. (2011). The ICOPER reference model for outcome-based higher education. Retrieved from http://www.icoper.org/results/deliverables/D7-3b. Accessed 27 Nov 2018.
- Tintarev, N. 2010. Explaining Recommendations. PhD Thesis, University of Aberdeen.Google Scholar
- Tintarev, N., & Masthoff, J. (2011). Designing and evaluating explanations for recommender systems. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), (pp. 479–510). Springer US. Retrieved from https://doi.org/10.1007/978-0-387-85820-3_15.
- Wills, S., & McDougall, A. (2009). Reusability of online role play: Learning objects or learning designs? In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of research on learning design and learning objects: Issues, applications, and technologies (p. ??). Information Science Reference. Retrieved from http://ro.uow.edu.au/asdpapers/96/. Accessed 27 Nov 2018.
- Yoo, K. H., & Gretzel, U. (2011). Creating more credible and persuasive recommender systems: the influence of source characteristics on recommender system evaluations. Recommender Systems Handbook, 455–477. https://doi.org/10.1007/978-0-387-85820-3.