Abstract
This paper describes a probabilistic model for optimum information retrieval in a distributed heterogeneous environment.
The model assumes the collection of documents offered by the environment to be partitioned into subcollections. Documents as well as subcollections have to be indexed, where indexing methods using different indexing vocabularies can be employed. A query provided by a user is answered in terms of a ranked list of documents. The model determines a procedure for ranking the documents that stems from the Probability Ranking Principle: For each subcollection, the subcollection's documents are ranked; the resulting ranked lists are combined into a final ranked list of documents, where the ordering is determined by the documents' probabilities of being relevant with respect to the user's query. Various probabilistic ranking methods may be involved in the distributed ranking process. A criterion for effectively limiting the ranking process to a subset of subcollections extends the model.
The property that different ranking methods and indexing vocabularies can be used is important when the subcollections are heterogeneous with respect to their content.
The model's applicability is experimentally confirmed. When exploiting the degrees of freedom provided by the model, experiments showed evidence that the model even outperforms comparable models for the non-distributed case with respect to retrieval effectiveness.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Baumgarten C (1997) A probabilistic model for distributed information retrieval. In: Proc. 20th ACMSIGIR Conf. on Research and Development in Information Retrieval.
Baumgarten C (1999a) Probabilistic information retrieval in a distributed heterogeneous environment. PhD Thesis, Dresden Univ. of Techn.
Baumgarten C (1999b) A probabilistic solution to the selection and fusion problem in distributed information retrieval. In: Proc. 22nd Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Berkeley, CA.
Callan JP, Zhihong L and Croft WB(1995) Searching distributed collections with inference networks. In: Proc. 18th ACM SIGIR Conf. on Research and Development in Information Retrieval.
French J, Powell A, Viles C, Emmitt T and Prey K (1998) Evaluating database selection techniques: A testbed and experiment. In: Proc. 21st ACM SIGIR Conf. on Research and Development in Information Retrieval.
Fuhr N (1992) Integration of probabilistic fact and text retrieval. In: Proc. 15th ACM SIGIR Conf. on Research and Development in Information Retrieval.
Fuhr N (1993) Information retrieval. Course material of the course held in the summer term 1993. University of Dortmund. Available at http://ls6.informatik.uni-dortmund.de/ir/teaching/courses/ir/.
Fuhr N (1999) A decision-theoretic approach to database selection in networked IR. ACM Transactions on Information Systems, 17(3).
Gravano L and Garcia-Molina H (1995) Generalizing GlOSS to vector-space databases and broker hierarchies. In: Proc. 21st VLDB Conf.
Harman D (1993) Overview of the first TREC conference. In: Proc. 16th ACM SIGIR Conf. on Research and Development in Information Retrieval.
Meng W, Liu K, Yu C, Wang X, Chang Y and Rishe N (1998) Determining text databases to search in the internet. In: Proc. 24th VLDB Conf.Extended version.
Porter M (1980) An algorithm for suffix stripping. Program 14.
Robertson S (1977) The probability ranking principle in IR. J. of Documentation, 33(4).
Robertson S and Sparck-Jones K(1976) Relevance weighting of search terms'. J. American Society for Inf. Science,27.
Salton G and McGill MJ (1983) Introduction to Modern Information Retrieval, McGraw-Hill, New York.
Schäuble P (1997) Multimedia Information Retrieval-Content-Based Information Retrieval from Large Text and Audio Databases, Kluwer Academic, Boston.
Stahel WA (1995) Statistische Datenanalyse, Vieweg Verlag, Braunschweig.
Voorhees EM, Gupta NK and Johnson-Laird B (1994) The collection fusion problem. In: Harman DK (Ed.), Proc. TREC-3.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Baumgarten, C. Retrieving Information from a Distributed Heterogeneous Document Collection. Information Retrieval 3, 253–271 (2000). https://doi.org/10.1023/A:1026572910743
Issue Date:
DOI: https://doi.org/10.1023/A:1026572910743