Advertisement

Using explanations for recommender systems in learning design settings to enhance teachers’ acceptance and perceived experience

  • Soultana KargaEmail author
  • Maya Satratzemi
Article
  • 48 Downloads

Abstract

The reuse of Learning Designs can bring significant advantages to the educational community such as the diffusion of best teaching practices and the improvement of teaching quality and learning outcomes. Although various tools, including Recommender Systems, have been developed to implement the notion of reusing Learning Designs, their adoption by teachers falls short of expectations. This paper investigates the results of providing explanations for Learning Design recommendations to teachers. To this end, we designed and implemented an explanatory mechanism incorporated into a Recommender System, which propose pre-existing Learning Designs to teachers. We then conducted a user-centric evaluation experiment. Overall, this study provides evidence that explanations should be incorporated into Recommender Systems that propose Learning Designs, as a way of improving the teacher-perceived experience and promoting their wider adoption by teachers. The more teachers accept and adopt Recommender Systems that propose Learning Designs, the more the educational community gains the benefits of reusing Learning Designs.

Keywords

Explanations Recommender systems Learning design Technology enhanced learning 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. Agostinho, S. (2009). Learning Design Representations to Document, Model, and Share Teaching Practice. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of Research on Learning Design and Learning Objects: Issues, Applications, and Technologies (pp. 1–19). Hershey: IGI Global.  https://doi.org/10.4018/978-1-59904-861-1.ch001.
  2. Agostinho, S., Bennett, S., Lockyer, L., Jones, J., & Harper, B. (2013). Learning designs as a stimulus and support for teachers’ design practices. In H. Beetham & R. Sharpe (Eds.), Rethinking Pedagogy for a Digital Age: Designing for 21st century (pp. 119–132). New York: Routledge.Google Scholar
  3. Beetham, H., & Sharpe, R. (2007). Rethinking pedagogy for a digital age: designing and delivering e-learning.  https://doi.org/10.4324/9780203961681.
  4. Bennett, S., Agostinho, S., & Lockyer, L. (2005). Reusable learning designs in university education. Proceedings of the IASTED International Conference on Education and Technology, ICET 2005, 2005, 102–106. Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-33751297745&partnerID=40&md5=de22754ab9711f446c2d71bc23695040. Accessed 27 Nov 2018.
  5. Bennett, S., Thomas, L., Agostinho, S., Lockyer, L., Jones, J., & Harper, B. (2011). Understanding the design context for Australian university teachers: implications for the future of learning design. Learning, Media and Technology, 36(2), 151–167.  https://doi.org/10.1080/17439884.2011.553622.Google Scholar
  6. Bennett, S., Agostinho, S., & Lockyer, L. (2017). The process of designing for learning: understanding university teachers’ design work. Educational Technology Research and Development, 65(1), 125–145.  https://doi.org/10.1007/s11423-016-9469-y.Google Scholar
  7. Cameron, L. (2009). How learning design can illuminate teaching practice. The Future of Learning Design Conference.Google Scholar
  8. Cameron, L., & Campbell, C. (2010). Sharing learning designs that work. In EdMedia: world Conference on Educational Media and Technology (pp. 1914–1919).Google Scholar
  9. Celik, D., & Magoulas, G. D. (2016). A review, timeline, and categorization of learning design tools. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 10013 LNCS, pp. 3–13).  https://doi.org/10.1007/978-3-319-47440-3_1
  10. Chen, L., & Wang, F. (2017). Explaining recommendations based on feature sentiments in product reviews. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces (pp. 17–28). New York: ACM.  https://doi.org/10.1145/3025171.3025173.
  11. Conole, G. (2013). Designing for learning in an open world. New York: Springer New York. Retrieved from  https://doi.org/10.1007/978-1-4419-8517-0
  12. Conole, G., & Culver, J. (2010). The design of Cloudworks: applying social networking practice to foster the exchange of learning and teaching ideas and designs. Computers in Education, 54(3), 679–692.  https://doi.org/10.1016/j.compedu.2009.09.013.Google Scholar
  13. Conole, G., & Fill, K. (2005). A learning design toolkit to create pedagogically effective learning activities. Journal of Interactive Media in Education, 8, 1–16.  https://doi.org/10.5334/2005-8.Google Scholar
  14. Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008). The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18(5), 455–496.  https://doi.org/10.1007/s11257-008-9051-3.Google Scholar
  15. Dalziel, J. (2003). Implementing learning design : the learning activity management system (LAMS). Retrieved from http://hdl.handle.net/1959.14/79594. Accessed 28 Nov 2018.
  16. Dalziel, J., Conole, G., Wills, S., Walker, S., Bennett ǁ, S., Dobozy, E., et al. (2016). The Larnaca declaration on learning design. Journal of Interactive Media in Education, 2016(7), 1–24.  https://doi.org/10.5334/jime.407.Google Scholar
  17. Donald, C., Blake, A., Girault, I., Datt, A., & Ramsay, E. (2009). Approaches to learning design: past the head and the hands to the HEART of the matter. Distance Education, 30(2), 179–199.  https://doi.org/10.1080/01587910903023181.Google Scholar
  18. Erdt, M., Fernandez, A., & Rensing, C. (2015). Evaluating recommender systems for technology enhanced learning: a quantitative survey. IEEE Transactions on Learning Technologies, 1382(c), 1.  https://doi.org/10.1109/TLT.2015.2438867.Google Scholar
  19. Fazeli, S., Drachsler, H., Bitter-Rijpkema, M., Brouns, F., van der Vegt, W., & Sloep, P. B. (2017). User-centric evaluation of recommender systems in social learning platforms: accuracy is just the tip of the iceberg. IEEE Transactions on Learning Technologies, 11, 294–306.  https://doi.org/10.1109/TLT.2017.2732349.Google Scholar
  20. Gedikli, F., Jannach, D., & Ge, M. (2014). How should I explain? A comparison of different explanation types for recommender systems. International Journal of Human-Computer Studies, 72(4), 367–382.  https://doi.org/10.1016/j.ijhcs.2013.12.007.Google Scholar
  21. Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000). Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (pp. 241–250). New York: ACM.  https://doi.org/10.1145/358916.358995
  22. Hernández-Leo, D., Harrer, A., Dodero, J. M., Asensio-Pérez, J. I., & Burgos, D. (2007). A framework for the conceptualization of approaches to create-by-reuse of learning design solutions. Journal of Universal Computer Science, 13(7), 750–760.Google Scholar
  23. Hernández-Leo, D., Chacón, J., Prieto, L. P., Asensio-Pérez, J. I., & Derntl, M. (2013). Towards an integrated learning design environment. In D. Hernández-Leo, T. Ley, R. Klamma, & A. Harrer (Eds.), Scaling up learning for sustained impact (pp. 448–453). Berlin: Springer Berlin Heidelberg.Google Scholar
  24. Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Prieto, L. P., & Chacón, J. (2014). ILDE: community environment for conceptualizing, authoring and deploying learning activities. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8719 LNCS, pp. 490–493).  https://doi.org/10.1007/978-3-319-11200-8_48
  25. Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Pozzi, F., Chacón, J., Prieto, L. P., & Persico, D. (2018). An integrated environment for learning design. Frontiers in ICT, 5(May), 1–19.  https://doi.org/10.3389/fict.2018.00009.Google Scholar
  26. Hinkin, T. R., Tracey, J. B., & Enz, C. A. (1997). Scale construction: developing reliable and valid measurement instruments. Journal of Hospitality and Tourism Research, 21(1), 100–120.  https://doi.org/10.1177/109634809702100108.Google Scholar
  27. Jin, X., Zhou, Y., & Mobasher, B. (2005). A maximum entropy web recommendation system : combining collaborative and content features. Intelligence, 24(6), 612–617.  https://doi.org/10.1145/1081870.1081945.Google Scholar
  28. Karga, S., & Satratzemi, M. (2018). A hybrid recommender system integrated into LAMS for learning designers. Education and Information Technologies, 23(3), 1297–1329.  https://doi.org/10.1007/s10639-017-9668-0.
  29. Koper, R., & Bennett, S. (2008). Learning design: Concepts. In H. Adelsberger, Kinshuk, J. Pawlowski, & D. Sampson (Eds.), (pp. 135–154). Springer Berlin Heidelberg. Retrieved from  https://doi.org/10.1007/978-3-540-74155-8_8.
  30. Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., & Getoor, L. (2017). User preferences for hybrid explanations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 84–88). New York: ACM.  https://doi.org/10.1145/3109859.3109915.
  31. Laurillard, D. (2012). Teaching as a design science. Building pedagogical patterns for learning and technology. Routledge: Taylor & Francis Group.Google Scholar
  32. Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E. A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15–30.  https://doi.org/10.1111/j.1365-2729.2011.00458.x.Google Scholar
  33. Lee, S. (2014). Rethinking pedagogy for a digital age: designing for twenty-first century learning. Open Learning: The Journal of Open, Distance and E-Learning, 29(2), 174–176.  https://doi.org/10.1080/02680513.2014.940498.Google Scholar
  34. Lockyer, L., Bennett, S., Agostinho, S., & Harper, B. (2009). Handbook of Research on Learning Design and Learning Objects. In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.). IGI Global.  https://doi.org/10.4018/978-1-59904-861-1.
  35. Maina, M., Craft, B., & Mor, Y. (2015). The Art & Science of Learning Design. Rotterdam: Sense Publishers. Retrieved from https://www.sensepublishers.com/catalogs/bookseries/technology-enhanced-learning-1/the-art-and-science-of-learning-design/. Accessed 27 Nov 2018.
  36. McNee, S., Lam, S., Konstan, J., & Riedl, J. (2003). Interfaces for Eliciting New User Preferences in Recommender Systems. User Modeling 2003, 2702, 148.  https://doi.org/10.1007/3-540-44963-9_24.
  37. McNee, S. M., Riedl, J., & Konstan, J. a. (2006). Being accurate is not enough: How accuracy metrics have hurt recommender systems. CHI’06 Extended Abstracts on Human Factors in Computing Systems, 1101.  https://doi.org/10.1145/1125451.1125659.
  38. Mor, Y., & Craft, B. (2012). Learning design: reflections on a snapshot of the current landscape. Research in Learning Technology, 20. Retrieved from http://www.researchinlearningtechnology.net/index.php/rlt/article/view/19196/. Accessed 27 Nov 2018.
  39. Muhammad, K. I., Lawlor, A., & Smyth, B. (2016). A live-user study of opinionated explanations for recommender systems. In Proceedings of the 21st International Conference on Intelligent User Interfaces (pp. 256–260). New York: ACM.  https://doi.org/10.1145/2856767.2856813.
  40. Muñoz, O. R., Penalba, F. A., & Sánchez, J. F. (2016). The skills, competences, and attitude toward information and communications technology recommender system: an online support program for teachers with personalized recommendations. New Review of Hypermedia and Multimedia, 22(1–2), 83–110.  https://doi.org/10.1080/13614568.2015.1036132.Google Scholar
  41. O’Donovan, J., Smyth, B., Gretarsson, B., Bostandjiev, S., & Höllerer, T. (2008). PeerChooser: visual interactive recommendation. In SIGCHI Conference on Human Factors in Computing Systems (pp. 1085–1088). Florence: ACM.  https://doi.org/10.1145/1357054.1357222.
  42. Paquette, G., Teja, I. D. la, & K. Lundgren-Cayrol. (2006). Learning design classification definitions. Retrieved from http://helios.licef.ca/residld/4/Classification_Design_P%C3%A9dagogique.doc. Accessed 27 Nov 2018.
  43. Prieto, L., Dimitriadis, Y., Craft, B., Derntl, M., Émin, V., Katsamani, M., … Villasclaras, E. (2013). Learning design Rashomon II: Exploring one lesson through multiple tools. Research in Learning Technology, 21(0).  https://doi.org/10.3402/rlt.v21i0.20057.
  44. Pu, P., & Chen, L. (2006). Trust building with explanation interfaces. In Proceedings of the 11th International Conference on Intelligent User Interfaces (pp. 93–100). New York: ACM.  https://doi.org/10.1145/1111449.1111475.
  45. Pu, P., & Chen, L. (2011). A user - centric evaluation framework for recommender systems. Proceedings of the 5th ACM Conference on Recommender Systems - RecSys ‘11, 157–164.  https://doi.org/10.1145/2043932.2043962.
  46. Pu, P., Faltings, B., Chen, L., Zhang, J., & Viappiani, P. (2011). Usability guidelines for product recommenders based on example critiquing research. In Recommender Systems Handbook (pp. 511–545).  https://doi.org/10.1007/978-0-387-85820-3_16.Google Scholar
  47. Quijano-Sanchez, L., Sauer, C., Recio-Garcia, J. A., & Diaz-Agudo, B. (2017). Make it personal: A social explanation system applied to group recommendations. Expert Systems with Applications, 76, 36–48.  https://doi.org/10.1016/j.eswa.2017.01.045.
  48. Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56–58.  https://doi.org/10.1145/245108.245121.Google Scholar
  49. Salter, J., & Antonopoulos, N. (2006). CinemaScreen recommander agent: Combining collaborative and content-based filtering. IEEE Intelligent Systems, 21, 35–41.  https://doi.org/10.1109/MIS.2006.4.Google Scholar
  50. Santos, O. C., & Boticario, J. G. (2011). Requirements for semantic educational recommender systems in formal e-learning scenarios. Algorithms, 4(2), 131–154.  https://doi.org/10.3390/a4030131.zbMATHGoogle Scholar
  51. Simon, B., Pulkkinen, M., Totschnig, M., & Kozlov, D. (2011). The ICOPER reference model for outcome-based higher education. Retrieved from http://www.icoper.org/results/deliverables/D7-3b. Accessed 27 Nov 2018.
  52. Symeonidis, P., Nanopoulos, A., & Manolopoulos, Y. (2008). Providing justifications in recommender systems. Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 38(6), 1262–1272.  https://doi.org/10.1109/TSMCA.2008.2003969.Google Scholar
  53. Tintarev, N. 2010. Explaining Recommendations. PhD Thesis, University of Aberdeen.Google Scholar
  54. Tintarev, N., & Masthoff, J. (2011). Designing and evaluating explanations for recommender systems. In F. Ricci, L. Rokach, B. Shapira, & P. B. Kantor (Eds.), (pp. 479–510). Springer US. Retrieved from  https://doi.org/10.1007/978-0-387-85820-3_15.
  55. Tintarev, N., & Masthoff, J. (2012). Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction, 22(4), 399–439.  https://doi.org/10.1007/s11257-011-9117-5.Google Scholar
  56. Tintarev, N., & Masthoff, J. (2015). Explaining recommendations: Design and evaluation. In Recommender Systems Handbook (2nd ed., pp. 353–382).  https://doi.org/10.1007/978-1-4899-7637-6_10.Google Scholar
  57. Verbert, K., Ochoa, X., Derntl, M., Wolpers, M., Pardo, A., & Duval, E. (2012). Semi-automatic assembly of learning resources. Computers & Education, 59(4), 1257–1272.  https://doi.org/10.1016/j.compedu.2012.06.005.Google Scholar
  58. Weller, M. (2004). Learning objects and the e-learning cost dilemma. Journal of Open, Distance and E-Learning, 19(3), 37–41.  https://doi.org/10.1080/0268051042000280147.MathSciNetGoogle Scholar
  59. Wills, S., & McDougall, A. (2009). Reusability of online role play: Learning objects or learning designs? In L. Lockyer, S. Bennett, S. Agostinho, & B. Harper (Eds.), Handbook of research on learning design and learning objects: Issues, applications, and technologies (p. ??). Information Science Reference. Retrieved from http://ro.uow.edu.au/asdpapers/96/. Accessed 27 Nov 2018.
  60. Wills, S., & Pegler, C. (2016). A deeper understanding of reuse: learning designs, activities, resources and their contexts. Journal of Interactive Media in Education, 2016(1), 1–11.  https://doi.org/10.5334/jime.405.Google Scholar
  61. Yoo, K. H., & Gretzel, U. (2011). Creating more credible and persuasive recommender systems: the influence of source characteristics on recommender system evaluations. Recommender Systems Handbook, 455–477.  https://doi.org/10.1007/978-0-387-85820-3.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Applied InformaticsUniversity of MacedoniaThessalonikiGreece

Personalised recommendations