Exploring the Effect of Query Translation when Searching Cross-Language
A usability study of Clarity, a cross language information retrieval system for rare languages, is presented. Clarity aims at investigating CLIR for so-called low-density languages, those with few translation resources. The usability study explored two different levels of feedback and control over the query translation mechanism. Techniques like word-by-word translation of title and keywords were also tested. Although it would appear that a greater control over query translation enables users to retrieve more relevant documents a great difference among participants, topics, and tasks was discovered. Indeed the user engagement with the searching task is extremely subjective and variable, thus affecting the homogeneity of the results and preventing any statistical validity. A revision of the current evaluation schema is important to get a better understanding of user-CLIR interaction and some issues on different ways of measuring user’s performance are outlined in this perspective.
Unable to display preview. Download preview PDF.
- Airio, E, Keskustalo, H., Hedlund, T., Pirkola, A.: UTACLIR @ CLEF 2002: Towards a unified translation process model. Working notes CLEF2002, (2002), 51-58.Google Scholar
- He, D., Wang, J., Oard, D., and Nossal, M.: Comparing User-assisted and Automatic Query Translation. Working notes CLEF 2002, (2002) 267-278.Google Scholar
- Koenemann, J., and Belkin, N.J.: A case for interaction: A study of interactive information retrieval behaviour and effectiveness. Proceedings of CHI96, (1996) 205-212Google Scholar
- Oard, D.: Serving users in many languages cross-language information retrieval for digital libraries, D-Lib Magazine, December 1997.Google Scholar
- Ogden, W. C., and Davis, M.W.: Improving cross-language text retrieval with human interactions. Proceedings of the Hawaii International Conference on System Science — HICSS-33, 2000.Google Scholar
- Petrelli, D., Hansen, P., Beaulieu, M. Sanderson M.: User Requirement Elicitation for Cross-Language Information Retrieval. Proceedings of International Conference on Information Seeking and Retrieval in Context — ISIC 2002, Lisbon, September 2002.Google Scholar