Using Concept Hierarchies in Text-Based Image Retrieval: A User Evaluation

  • Daniela Petrelli
  • Paul Clough
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4022)


This paper describes our results from the image retrieval task of iCLEF 2005 based on a comparative user evaluation of two interfaces: one displaying search results as a list; the other organising retrieved images into a hierarchy of concepts displayed on the interface as an interactive menu. Based on a known-item retrieval task, data was analysed with respect to effectiveness, efficiency and user satisfaction. Effectiveness and efficiency were calculated at both the set cut-off time of 5 minutes, and the time after finding the target image (final time). Results showed the list was marginally more effective than the menu at 5 minutes, but the two were equal at final time indicating the menu requires more time to be used effectively. The list was more efficient at both 5 minutes and final time (difference not statistically significant) and users preferred using the menu indicating this could be a potentially interesting and engaging feature for image retrieval.


Image Retrieval Final Time User Evaluation User Satisfaction Concept Hierarchy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hearst, M.: User Interfaces and Visualization. In: Baeza-Yates, R., Ribeiro-Neto, B. (eds.) Modern Information Retrieval, pp. 257–323. ACM Press, New York (1999)Google Scholar
  2. 2.
    Sanderson, M., Croft, B.: Deriving concept hierarchies from text. In: Proceedings of the 22nd ACM Conference of the Special Interest Group in Information Retrieval, pp. 206–213 (1999)Google Scholar
  3. 3.
    Joho, H., Sanderson, M., Beaulieu, M.: A Study of User Interaction with a Concept-based Interactive Query Expansion Support Tool. In: McDonald, S., Tait, J. (eds.) Advances in Information Retrieval, 26th European Conference on Information Retrieval, pp. 42–56 (2004)Google Scholar
  4. 4.
    Savoy, J., Berger, P.-Y.: Selection and Merging Strategies for Multilingual Information Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 27–37. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Robertson, S.E., Walker, S., Beaulieu, M.M., Gatford, M., Payne, A.: Okapi at TREC-4. In: Harman, D.K. (ed.) NIST Special Publication 500-236: The Fourth Text REtrieval Conference (TREC-4), Gaithersburg, MD, pp. 73–97 (1995)Google Scholar
  6. 6.
    Petrelli, D., Beaulieu, M., Sanderson, M., Demetriou, G., Herring, P.: Observing Us-ers Designing Clarity: A Case Study on the User-Centered Design of a Cross-Language In-formation Retrieval System. JASIST Journal of the American Society for Information Sci-ence and Technology 55(10), 923–934 (2004)CrossRefGoogle Scholar
  7. 7.
    Petrelli, D., Levin, S., Beaulieu, M., Sanderson, M.: Which User Interaction for Cross-Language IR? Design Issues and Reflections. JASIST special issue on Multilingual Information Access 57(5), 709–722 (2005)Google Scholar
  8. 8.
    Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human-computer interface. In: CHI 1998, pp. 213–218. ACM Press, New York (1998)Google Scholar
  9. 9.
    Gonzalo, J., Clough, P., Vallin, A.: Overview of the CLEF 2005 Interactive Track. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 251–262. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Van Welie, M., van der Veer, G.C., Eliens, A.: Breaking down Usability. In: Proc. INTERAC 1999, pp. 613–620 (1999)Google Scholar
  11. 11.
    Frokjaer, E., Hertzum, M., Hornbaek, K.: Measuring Usability: Are Effectiveness, Efficiency, and User Satisfaction Really Correlated? In: CHI 2000, pp. 345–352 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Daniela Petrelli
    • 1
  • Paul Clough
    • 1
  1. 1.Department of Information StudiesUniversity of SheffieldSheffieldUK

Personalised recommendations