Supporting the discoverability of open educational resources
- 89 Downloads
Open Educational Resources (OERs), now available in large numbers, have a considerable potential to improve many aspects of society, yet one of the factors limiting this positive impact is the difficulty to discover them. This study investigates and proposes strategies to better support educators in discovering OERs, mainly focusing on secondary education. The literature suggests that the effectiveness of existing search systems could be improved by supporting high-level and domain-oriented tasks. Hence a preliminary taxonomy of discovery-related tasks was developed, based on the analysis of the literature, interpreted through Information Foraging Theory. This taxonomy was empirically evaluated with a few experienced educators, to preliminary identify an interesting class of Query By Examples (QBE) expansion by similarity tasks, which avoids the need to decompose natural high-level tasks in a complex sequence of sub-tasks. Following the Design Science Research methodology, three prototypes to support as well as to refine those tasks were iteratively designed, implemented, and evaluated involving an increasing number of educators in usability oriented studies. The resulting high-level and domain-oriented blended search/recommendation strategy, transparently replicates Google searches in specialized networks, and identifies similar resources with a QBE strategy. It makes use of a domain-oriented similarity metric based on shared schema.org/LRMI alignments to educational frameworks, and clusters results in expandable classes of comparable degree of similarity. The summative evaluation shows that educators appreciate this exploratory-oriented strategy because – balancing similarity and diversity – it supports their high-level tasks, such as lesson planning and personalization of education.
KeywordsDiscoverability Exploratory search OER schema.org Educational alignments
We would like to thank Prof. Marian Petre for her advice and feedback.
Compliance with ethical standards
The research was reviewed by, and received a favourable opinion from, the Open University Human Research Ethics Committee.
Conflict of interest
The authors declare that they have no conflict of interest.
- Al-Khalifa, H. S., & Davis, H. C. (2006). The evolution of metadata from standards to semantics in E-learning applications. In Proceedings of the seventeenth conference on Hypertext and hypermedia (pp. 69-72). New York: ACM.Google Scholar
- Atenas, J., & Havemann, L. (2013). Quality assurance in the open: An evaluation of OER repositories. INNOQUAL: International Journal for Innovation and Quality in Learning, 1(2), 22–34.Google Scholar
- Atkins, D. E., Brown, J. S., & Hammond, A. L. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities. Menlo Park: The William and Flora Hewlett Foundation.Google Scholar
- Barker, P. (2014). Explaining the LRMI alignment object. Sharing and learning. Resource document. http://blogs.pjjk.net/phil/explaining-the-lrmi-alignment-object/. Accessed 2 June 2018.
- Barker, P., & Campbell, L. M. (2016a). Technology strategies for open educational resource dissemination. In Open education: international perspectives in higher education (pp. 51–70). Cambridge: Openbook Publishers.Google Scholar
- Barker, P., & Campbell, L. (2016b). Learning resource metadata on the web. In International conference on world wide web (WWW ‘15 companion) (pp. 687–688). https://doi.org/10.1145/2740908.2741745.
- Belkin, N. J. (1995). Strategies for evaluation of interactive multimedia information retrieval systems. Proceedings of the final workshop on multimedia information retrieval (MIRO 95), Glasgow, London: Springer.Google Scholar
- Berners-Lee, T. (2007). Giant global graph. timbl’s blog. Resource document. http://dig.csail.mit.edu/breadcrumbs/node/215. Accessed 10 February 2014.
- Brooke, J. (1986). System usability scale (SUS): A quick-and-dirty method of system evaluation user information. Reading UK: Digital Equipment Co Ltd..Google Scholar
- Byström, K., & Hansen, P. (2002). Work tasks as unit for analysis in information seeking and retrieval studies. In Proceedings of the 4th international conference on conceptions of library and information science: emerging frameworks and methods (CoLIS4) (pp. 239–252). Greenwood Village: Libraries Unlimited.Google Scholar
- Chi, E. H. (2015). Blurring of the boundary between interactive search and recommendation. Proceedings of the 20th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/2678025.2700998.
- Cho, J. Y., & Lee, E. (2014). Reducing confusion about grounded theory and qualitative content analysis: Similarities and differences. The Qualitative Report, 19(32), 1–20.Google Scholar
- Cleverdon, C. W. (1960). ASLIB Cranfield research project on the comparative efficiency of indexing systems. ASLIB Proceedings, XII, 421–431.Google Scholar
- D’Aquin, M., Adamou, A., & Dietze, S. (2013). Assessing the educational linked data landscape. Proceedings of the 5th annual ACM Web science conference (pp. 43–46). New York: ACM.Google Scholar
- Daher, J., Brun, A., & Boyer, A. (2017). A review on explanations in recommender systems. Technical Report LORIA - Université de Lorraine.Google Scholar
- DCMI (2012). DC-education application profile. Resource document. http://dublincore.org/moinmoin-wiki-archive/educationwiki/pages/DC_2dEducation_20Application_20Profile.html. Accessed 2 May 2016.
- Dietze, S., Drachsler, H., & Giordano, D. (2014). A survey on linked data and the social web as facilitators for TEL recommender systems. In Recommender systems for technology enhanced learning (pp. 47–75). New York: Springer.Google Scholar
- Dietze, S., Taibi, D., Yu, R., Barker, P., & d’Aquin, M. (2017). Analysing and improving embedded markup of learning resources on the web. In Proceedings of the 26th international conference on world wide web companion (pp. 283–292). Republic and Canton of Geneva: International World Wide Web Conferences Steering Committee.Google Scholar
- Doctorow, C. (2001). Metacrap: Putting the torch to seven straw-men of the meta-utopia. Resource document. http://www.well.com/doctorow/metacrap.htm. Accessed 15 April 2015.
- Dou, Z., Song, R., & Wen, J. R. (2007, May). A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th international conference on World Wide Web, 581-590. ACM.Google Scholar
- Downes, S. (2003). One standard for all: Why We Don’t Want It, Why We Don’t Need It. National Research Council. Resource document. http://zope.cetis.ac.uk/lib/media/one_standard.pdf. Accessed 2 May 2016.
- Drachsler, H., Greller, W., Fazeli, S., Niemann, K., Sanchez-Alonso, S., Rajabi, E., Palmér, M., Ebner, H., Simon, B., Nösterer, D., Kastrantas, K., Manouselis, N., Hatzakis, I., & Clements, K. (2012). D8.1 review of social data requirements. Open Discovery Space project.Google Scholar
- Dziadosz, S., & Chandrasekar, R. (2002). Do thumbnail previews help users make better relevance decisions about web search results? In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 365–366). New York: ACM.Google Scholar
- GLOBE (2016). GLOBE – Connecting the world and unlocking the deep web. Resource document. http://globe-info.org. Accessed 30 July 2016.
- Gurell, S., & Wiley, D. (2008). OER handbook for educators. Resource document. http://wikieducator.org/OER_Handbook/educator_version_one. Accessed 11 Aug 2017.
- IEEE (2002). Draft standard for learning object metadata. Resource document. http://ltsc.ieee.org/wg12/files/LOM_1484_12_1_v1_Final_Draft.pdf. Accessed 10 April 2014.
- IMS (2015). Interoperability standards helping to transform the digital curriculum. Resource document. www.imsglobal.org/sites/default/files/articles/slI17-051815.pdf. Accessed 10 June 2015.
- ISO. (1998). ISO 9241-11:1998: Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability. Geneva: International Organization for Standardization.Google Scholar
- Kearney, M. (2017). Measure performance with the RAIL model. Resource document. https://developers.google.com/web/fundamentals/performance/rail. Accessed 13 August 2017.
- Klo, J. (2011). AMPlied search extension for chrome. Resource document. https://github.com/jimklo/AMPS-Chrome. Accessed 01 May 2016.
- Knoth, P. (2015). Linking Textual Resources to Support Information Discovery. Ph.D. thesis. Milton Keynes: The Open University.Google Scholar
- Learning Registry. (2016). Learning Registry. Resource document. www.learningregistry.org. Accessed 1 May 2016.
- Lockley, P. (2011). Pgogy/learning_registry_chrome. Resource document. https://github.com/pgogy/learning_registry_chrome. Accessed 1 May 2016.
- LRMI (2013a). The smart Publisher’s guide to LRMI tagging. Resource document. http://www.lrmi.net/wp-content/uploads/2013/03/LRMI_tagging_Guide.pdf. Accessed 5 March 2014.
- LRMI (2013b). LRMI survey report. Resource document. http://www.lrmi.net/wp-content/uploads/2013/08/LRMI-Survey-Report-August-2013-Update.pdf. Accessed 5 April 2014.
- LRMI (2013c). LRMI’s “killer feature”: educationalAlignment. Resource document. http://lrmi.dublincore.net/2013/06/12/lrmis-killer-feature-educationalalignment/. Accessed 2 May 2016.
- LRMI (2014). LRMI Project Description Resource document. http://wiki.lrmi.net/LRMI+Project+Description. Accessed 5 April 2014.
- Madan, A., & Dubey, S.K. (2012). Usability evaluation methods: A literature review. International Journal of Engineering Science and Technology, 4(2), 590–599.Google Scholar
- Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., & Koper, R. (2011). Recommender systems in technology enhanced learning. In Recommender systems handbook (pp. 387–415). New York: Springer.Google Scholar
- Nilsson, M. (2010). From interoperability to harmonization in metadata standardization: designing an evolvable framework for metadata harmonization. Ph.D. thesis. Stockholm: KTH.Google Scholar
- Prat, N., Comyn-Wattiau, I., & Akoka, J. (2014). Artifact evaluation in information systems design-science research-a holistic view. In PACIS 2014 Proceedings (vol. Paper 23, pp. 1–16).Google Scholar
- Riley, J. (2010). Glossary of metadata standards. Resource document. http://www.dlib.indiana.edu/~jenlrile/metadatamap/seeingstandards_glossary_pamphlet.pdf. Accessed 02 May 2016.
- Schema.org. (2013). Frequently asked questions. Resource document. http://schema.org/docs/faq.html. Accessed 10 June 2015.
- Shear, L., Means, B., & Lundh, P. (2015). Research on open: OER research hub review and futures for research on OER. Menlo Park: SRI International.Google Scholar
- Singhal, A. (2012). Introducing the knowledge graph: Things, not strings. Official Google Blog. Resource document. http://googleblog.blogspot.co.uk/2012/05/introducing-knowledge-graph-things-not.html. Accessed 13 April 2015.
- Smyth, B., & McClave, P. (2001). Similarity vs. diversity. In Case-based reasoning research and development (pp. 347–361). Berlin, Heidelberg: Springer.Google Scholar
- Sue, V. M., & Ritter, L. A. (2007). Designing and developing the survey instrument. In Conducting online surveys (pp. 59–88). Thousand Oaks: Sage.Google Scholar
- Sutton, S. A., & Mason, J. (2001). The Dublin Core and metadata for educational resources. In International conference on Dublin Core and metadata applications, Wroclaw, Poland (pp. 25–31).Google Scholar
- Timpany, G. (2015). Reaching the constant sum. Resource document. http://survey.cvent.com/blog/customer-insights-2/using-constant-sum-questions. Accessed 2 May 2016.
- UNESCO (2012). UNESCO Paris OER declaration. Resource document. http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf. Accessed 8 February 2014.
- Vaishnavi, V., & Kuechler, W. (2015). Design science research in information systems. Resource document. http://www.desrist.org/design-research-in-information-systems/. Accessed 5 May 2017.
- Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., & Duval, E. (2011). Dataset-driven research for improving recommender systems for learning. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 44–53). New York: ACM.Google Scholar
- Weller, M. (2010). Big and little OER. OpenED2010: Seventh Annual Open Education Conference, 2-4 Nov 2010, Barcelona, Spain. http://oro.open.ac.uk/24702/.
- White, R. W., Drucker, S. M., Marchionini, G., & Hearst, M. (2007). Exploratory search and HCI: Designing and evaluating interfaces to support exploratory search interaction. In CHI’07 extended abstracts on human factors in computing systems (pp. 2877–2880). New York: ACM.Google Scholar
- Wildemuth, B. M. & Freund, L. (2009). Search tasks and their role in studies of search behaviors. In Third annual workshop on human computer interaction and information retrieval, Washington DC.Google Scholar