User-Centered Evaluation of a Discovery Layer System with Google Scholar
Discovery layer systems allow library users to obtain search results from multiple library resources and view results in a consistent format. The implementation of a discovery layer is expected to simplify users’ workflow of searching for scholarly information. Previous studies on discovery layer systems focused on functionality and content, but not quality of search results from the user’s perspective. The objective of this study was to obtain users’ assessment of search results of a discovery layer system (Ex Libris Primo®) and compare that with a widely used scholarly search tool (Google Scholar). Results showed that Primo’s search results relevancy is comparable to Google Scholar, but it received significantly lower usability and preference ratings. A number of usability issues of Primo were also identified from the study. Results of the study are used to improve the interface of Primo and adjust relevancy ranking options. The empirical method of search results assessment and feedback collection used in this study can be extended to similar user-centered system implementation and evaluation efforts.
KeywordsDiscovery layer Google Scholar search results relevance user-centered evaluation
Unable to display preview. Download preview PDF.
- 4.Howland, J.L., Wright, T.C., Boughan, R.A., Roberts, B.C.: How Scholarly is Google Scholar? A Comparison to Library Databases. College & Research Libraries 70, 227–234 (2009)Google Scholar
- 8.Herrera, G.: Google Scholar Users and User Behaviors: An Exploratory Study. College & Research Libraries 72, 316–330 (2011)Google Scholar
- 10.Ward, J.L., Shadle, S., Mofjeld, P.: User Experience, Feedback, and Testing. Library Technology Reports 44, 17–23 (2008)Google Scholar
- 11.Brooke, J.: SUS - A Quick and Dirty Usability Scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor & Francis, London (1996)Google Scholar