A User Study on Features Supporting Subjective Relevance for Information Retrieval Interfaces

  • Shu-Shing Lee
  • Yin-Leng Theng
  • Dion Hoe-Lian Goh
  • Schubert Shou-Boon Foo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4312)


Objective relevance regards retrieved documents as relevant without considering users’ tasks. Subjective relevance, in contrast, focuses on usefulness of documents for users’ contexts. This paper aims to enhance objective relevance and address its limitations by designing a mock-up interface based on ACM Digital Library so as to enhance its support for users’ evaluating the subjective relevance of documents. Important features elicited using a factor analytic approach from an earlier study were used to inform the design of the search, results list and document record pages in the mock-up. A pilot study was conducted to gather users’ feedback about usefulness of designed features. Findings indicated that majority of important features designed were useful. However, subjects suggested that the design of some features could be further improved to facilitate their support for SR. The paper concludes by discussing recommendations for improving the mock-up.


Full Text Digital Library User Study Extended Format Boolean Operator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barry, C.L.: User-defined relevance criteria: an exploratory study. Journal of the American Society for Information Science 45(3), 149–159 (1994)CrossRefGoogle Scholar
  2. 2.
    Belkin, N.J., Oddy, R.N., Brooks, H.: ASK for information retrieval: Part I. background and theory. The Journal of Documentation 38(2), 61–71 (1982)CrossRefGoogle Scholar
  3. 3.
    Cosijin, E., Ingwersen, P.: Dimensions of relevance. Information Processing and Management 63, 533–550 (2000)CrossRefGoogle Scholar
  4. 4.
    Borlund, P., Ingwersen, P.: Measures for relative relevance and ranked half-life: Performance indicators for interactive IR. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 324–331. ACM Press, New York (1998)CrossRefGoogle Scholar
  5. 5.
    Druin, A., et al.: Designing a digital library for young children: An intergenerational partnership. In: Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 398–401. ACM Press, New York (2001)CrossRefGoogle Scholar
  6. 6.
    Ingwersen, P., Borlund, P.: Information transfer viewed as interactive cognitive processes. In: Ingwersen, P., Pors, N.O. (eds.) Information Science: Integration in Perspective, Royal School of Librarianship, Denmark, pp. 219–232 (1996)Google Scholar
  7. 7.
    Lee, S.-S., Theng, Y.-L., Goh, H.-L.D., Foo, S.-B.S.: An exploratory factor analytic approach to understand design features for academic learning environments. In: Gonzalo, J., Thanos, C., Verdejo, M.F., Carrasco, R.C. (eds.) ECDL 2006. LNCS, vol. 4172, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Meyyapan, N., Chowdhury, G.G., Foo, S.: Use of a digital work environment prototype to create a user-centered university library. Journal of Information Science 27(4), 249–264 (2001)Google Scholar
  9. 9.
    Mizzaro, S.: How many relevances in information retrieval? Interacting with Computers 10, 303–320 (1998)CrossRefGoogle Scholar
  10. 10.
    Shneiderman, B.: Designing the user interface: Strategies for effective human-computer interaction. Addison Wesley Longman, USA (1998)Google Scholar
  11. 11.
    Tang, R., Soloman, P.: Toward an understanding of the dynamics of relevance judgment: An analysis of one person’s search behavior. Information Process and Management 34, 237–256 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shu-Shing Lee
    • 1
  • Yin-Leng Theng
    • 1
  • Dion Hoe-Lian Goh
    • 1
  • Schubert Shou-Boon Foo
    • 1
  1. 1.Division of Information Studies, School of Communication and InformationNanyang Technological UniversitySingapore

Personalised recommendations