Experiments with Document Archive Size Detection
The size of a document archive is a very important parameter for resource selection in distributed information retrieval systems. In this paper, we present a method for automatically detecting the size (i.e. number of documents) of a document archive, in case the archive itself does not provided such information. In addition, a method for detecting the incremental change of the archive size is also presented, which can be useful for deciding if a resource description has become obsolete and needs to be regenerated. An experimental evaluation of these methods shows that they provide quite accurate information.
KeywordsLanguage Model Financial Time Wall Street Journal Incremental Change Information Retrieval System
Unable to display preview. Download preview PDF.
- K. Bharat and A. Z. Broder. A Technique for Measuring the Relative Size and Overlap of Public Web Search Engines. In Proceedings of 7th WWW Conference, Brisbane, Australia, April 1998.Google Scholar
- J. K. Callan, M. Connell, and A. Du. Automatic Discovery of Language Models for Text Databases. In Proceedings of ACM SIGMOD International Conference, pages 479–490, Philadelphia, USA, May 1999.Google Scholar
- J. K. Callan, Z. Lu, and W. Croft. Searching Distributed Collections with Inference Networks. In Proceedings of the 18th Annual International ACM SIGIR Conference, pages 21–28, Seattle, USA, June 1995.Google Scholar
- L. Gravano and H. García-Molina. Generalizing GlOSS to Vector-Space Database and Broker Hierarchies. In Proceedings of 21st VLDB International Conference, pages 78–89, Zürich, Switzerland, 1995.Google Scholar
- K. Liu, C. Yu, and W. Meng. Discovering the Representative of a Search Engine. In Proceedings of the ACM CIKM International Conference, pages 652–654, Mclean, Virginia, USA, November 2002.Google Scholar
- P. Ogilvie and J. Callan. Experiments using the Lemur Toolkit. In Proceedings of the 2001 Text Retrieval Conference (TREC), pages 103–108, Gaithersburg, Maryland, USA, November 2001.Google Scholar
- S. Wu and F. Crestani. Multi-objective Resource Selection in Distributed Information Retrieval. In Proceedings of IPMU02, International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, pages 1171–1178, Annecy, France, July 2002.Google Scholar
- B. Yuwono and D. Lee. Server Ranking for Distributed Test Retrieval Systems on the Internet. In Proceedings of the Fifth International Conference on Database Systems for Advanced Application, pages 41–50, Melbourne, Australia, April 1997.Google Scholar