User-Centered Evaluation of a Discovery Layer System with Google Scholar

  • Tao Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8015)


Discovery layer systems allow library users to obtain search results from multiple library resources and view results in a consistent format. The implementation of a discovery layer is expected to simplify users’ workflow of searching for scholarly information. Previous studies on discovery layer systems focused on functionality and content, but not quality of search results from the user’s perspective. The objective of this study was to obtain users’ assessment of search results of a discovery layer system (Ex Libris Primo®) and compare that with a widely used scholarly search tool (Google Scholar). Results showed that Primo’s search results relevancy is comparable to Google Scholar, but it received significantly lower usability and preference ratings. A number of usability issues of Primo were also identified from the study. Results of the study are used to improve the interface of Primo and adjust relevancy ranking options. The empirical method of search results assessment and feedback collection used in this study can be extended to similar user-centered system implementation and evaluation efforts.


Discovery layer Google Scholar search results relevance user-centered evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Williams, S.C., Foster, A.K.: Promise Fulfilled? An EBSCO Discovery Service Usability Study. Journal of Web Librarianship 5, 179–198 (2011)CrossRefGoogle Scholar
  2. 2.
    Fagan, J.C., Mandernach, M., Nelson, C.S., Paulo, J.R., Saunders, G.: Usability Test Results for Discovery Tool in an Academic Library. Information Technology and Libraries 31, 83–112 (2012)CrossRefGoogle Scholar
  3. 3.
    Meier, J.J., Conkling, T.W.: Google Scholar’s Coverage of the Engineering Literature: An Empirical Study. The Journal of Academic Librarianship 34, 196–201 (2008)CrossRefGoogle Scholar
  4. 4.
    Howland, J.L., Wright, T.C., Boughan, R.A., Roberts, B.C.: How Scholarly is Google Scholar? A Comparison to Library Databases. College & Research Libraries 70, 227–234 (2009)Google Scholar
  5. 5.
    Callicott, B., Vaughn, D.: Google Scholar vs. Library Scholar: Testing the Performance of Schoogle. Internet Reference Services Quarterly 10, 71–88 (2005)CrossRefGoogle Scholar
  6. 6.
    Neuhaus, C., Neuhaus, E., Asher, A., Wrede, C.: The Depth and Breadth of Google Scholar: An Empirical Study. Portal: Libraries and the Academy 6, 127–141 (2006)CrossRefGoogle Scholar
  7. 7.
    Walters, W.H.: Google Scholar Search Performance: Comparative Recall and Precision. Portal: Libraries and the Academy 9, 5–24 (2009)CrossRefGoogle Scholar
  8. 8.
    Herrera, G.: Google Scholar Users and User Behaviors: An Exploratory Study. College & Research Libraries 72, 316–330 (2011)Google Scholar
  9. 9.
    Becher, M., Schmidt, K.: Taking Discovery Systems for a Test Drive. Journal of Web Librarianship 5, 199–219 (2011)CrossRefGoogle Scholar
  10. 10.
    Ward, J.L., Shadle, S., Mofjeld, P.: User Experience, Feedback, and Testing. Library Technology Reports 44, 17–23 (2008)Google Scholar
  11. 11.
    Brooke, J.: SUS - A Quick and Dirty Usability Scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor & Francis, London (1996)Google Scholar
  12. 12.
    Mussell, J., Croft, R.: Discovery Layers and the Distance Student: Online Search Habits of Students. Journal of Library & Information Services in Distance Learning 7, 18–39 (2013)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tao Zhang
    • 1
  1. 1.Purdue University LibrariesWest LafayetteUSA

Personalised recommendations