Abstract
This is the first in the series of the papers on an application of the systems analytic approach to evaluation of information retrieval (IR). At the beginning, the importance of consistent terminology in information retrieval is discussed, and different definitions of “information retrieval” and “IR system” are presented. The importance of identifying boundaries of the IR system for clarifying terminological differences is discussed. The terminological problems in IR are summarized, and some of the existing approaches to their solution are identified. A historical overview of IR research is presented in order to explain the existing terminological differences and to create a common basis for a discussion of evaluation approaches to IR.
Similar content being viewed by others
REFERENCES
Cleverdon, C. W., The significance of the Cranfield tests on index languages. ACM SIGIR'91, pp. 3–12, 1991.
Salton, G., and McGill, M. J., Introduction to Modern Information Retrieval, McGraw-Hill, New York, 1983.
Salton, G., The state of retrieval system evaluation. IPM 28(4):441–449, 1992.
Lancaster, F. W., and Warner, A. J., Information Retrieval Today, Information Resources Press, Arlington, VA, 1993.
Hersh, W. R., Information Retrieval: A Health Care Perspective, Springer-Verlag, New York, 1996.
Wilson, P., Situational relevance. ISR 9:457–471, 1973.
Swanson, D. R., Information retrieval as a trial-and-error process. LQ 47:128–148, 1977.
Fidel, R., and Soergel, D., Factors affecting online bibliographic retrieval: A conceptual framework for research. JASIS 34(3):163–180, 1983.
Belkin, N. J., Cognitive models and information retrieval. Soc. Sci. Inf. Stud. 4:111–129, 1984.
Meadow, C. T., Problems of information science research–An opinion paper. Can. J. Inform. Sci. 11:18–23, 1986.
Cuadra, C. A., and Katter, R. V., Experimental Studies of Relevance Judgments. Vol. I: Project Summary. Final Report, System Development Corp. Santa Monica, CA, 1967. Report No. TM-3520/001/00.
Cuadra, C. A., and Katter, R. V., Opening the black box of “relevance.” JD 23(4):291–303, 1967.
Lesk, M. E., and Salton, G., Relevance assessments and retrieval system evaluation. ISR 4:343–359, 1969.
Saracevic, T., The concept of “relevance” in information science: A historical review. In Saracevic, T., (ed.), Introduction to Information Science, R. R. Bowker, New York, pp. 111–151, 1970.
Harter, S. P., The Cranfield II relevance assessments: A critical evaluation. LQ 41:229–243, 1971.
Saracevic, T., Relevance: A review of and a framework for the thinking on the notion in information science. JASIS 26:321–343, 1975.
Meadow, C. T., ‘Relevance?’. JASIS 36:354–355, 1985.
Swanson, D. R., Subjective versus objective relevance in bibliographic retrieval systems. LQ 56:389–398, 1986.
Eisenberg, M. B., and Schamber, L., Relevance: The search for a definition. ASIS Proceedings. 1988, Atlanta, GA, pp. 164–168.
Schamber, L., Eisenberg, M. B., and Nilan, M. S., A re-examination of relevance: Toward a dynamic, situational definition. IPM 26:755–776, 1990.
Harter, S. P., Psychological relevance and information science. JASIS 43:602–615, 1992.
FroehlichT. J., Relevance reconsidered—Towards an agenda for the 21st century: Introduction to special topic issue on relevance Research. JASIS 45(3):124–134, 1994.
Hersh, W., Relevance and retrieval evaluation: Perspectives from medicine. JASIS 45(3):201–206, 1994.
Schamber, L., Relevance and information behavior. ARIST, p. 3–48, 1994.
Mizzaro S., Relevance: The whole history. JASIS 48(9):810–832, 1997.
Fidel, R., and Crandall, M., Users' perception of the performance of a filtering system. SIGIR 198–205, 1997.
Belkin, N. J., Oddy, R. N., and Brooks, H. M., ASK for information retrieval: Part I. Background and theory. Part II. Results of design study. JD 38:6171, 145–164, 1982.
Robertson, S. E., and Hancock-Beaulieu, M. M., On the evaluation of IR systems. IPM 28(4):457–466, 1992.
Tague-Sutcliffe, J. M., The pragmatics of information retrieval experimentation, revisited. IPM 28(4):467–490, 1992.
Ogden, C. K., and Richards, I. A., The Meaning of Meaning, 8th edn., Harcourt, Brace and World, New York, 1946.
Scherrer, J.R., Concepts, knowledge and language in healthcare information systems: Follow-up 30 months later. Methods Inf. Med. 37(4/5):312–314, 1998.
Meadow, C. T., Text Information Retrieval Systems. 1st edn. Academic Press, San Diego, CA, 1992.
Cleverdon, C. W., and Thorne, R. G., a brief experiment with the Uniterm system of coordinate indexing for the cataloging of structural data. RAE Library Memorndum#7, AD35004. Farnborough, UK: Royal aircraft Establishment, 1954.
Harter, S. P., and Hert, C. A., Evaluation of information retrieval systems: Approaches, issues, and methods. Williams, M. E., (ed.), ARIST. Information Today, Medford, NJ, pp. 3–94, 1997.
Croft, W. B., Knowledge-based and statistical approaches to text retrieval. IEEE Expert 8:8–12, 1993.
Tague-Sutciffe, J. M., Some perspectives on the evaluation of information retrieval systems. JASIS 47(1):1–3, 1996.
Minker, J., Information storage and retrieval—A survey and functional description. SIGIR Forum, Assoc. Comp. Mach. 12(2): 1–108, 1977.
Mannoni, B., Bringing museums online. Commun. ACM 39(6):100–105, 1996.
Schatz, B., and Chen, H., Digital libraries: Technological advances and social impacts. Computer 32(2):45–50, 1999.
Tague, J., Salminen, A., and McClellan, C., Complete formal model for information retrieval systems. In ACM SIGIR International Conference on Research and Development in Information Retrieval; 1991 October, Chicago, IL: ACM Press, New York, pp. 14–20, 1991.
Borgman, C. L., Hirsh, S. G., and Hiller, J., Rethinking online monitoring methods for information retrieval systems: From search product to search process. JASIS 47(7):568–583, 1996.
Buckland, M., and Plaunt, C., On the construction of selection systems. Library Hi Tech. 4(12):15–28, 1994.
Shneiderman, B., Designing the User Interface: Strategies for Effective Human–Computer Interaction. Addison-Wesley, Reading, MA, 1992.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kagolovsky, Y., Moehr, J.R. Terminological Problems in Information Retrieval. Journal of Medical Systems 27, 399–408 (2003). https://doi.org/10.1023/A:1025687220609
Issue Date:
DOI: https://doi.org/10.1023/A:1025687220609