A Hybrid Approach to Index Maintenance in Dynamic Text Retrieval Systems

  • Stefan Büttcher
  • Charles L. A. Clarke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3936)


In-place and merge-based index maintenance are the two main competing strategies for on-line index construction in dynamic information retrieval systems based on inverted lists. Motivated by recent results for both strategies, we investigate possible combinations of in-place and merge-based index maintenance. We present a hybrid approach in which long posting lists are updated in-place, while short lists are updated using a merge strategy. Our experimental results show that this hybrid approach achieves better indexing performance than either method (in-place, merge-based) alone.


Main Memory Query Time Query Performance Index Construction Inverted List 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Büttcher, S., Clarke, C.L.A.: Indexing Time vs. Query Time Trade-offs in Dynamic Information Retrieval Systems. In: Proc. of the 14th ACM Conf. on Information and Knowledge Management (CIKM 2005), Bremen, Germany (2005)Google Scholar
  2. 2.
    Büttcher, S., Clarke, C.L.A.: Memory Management Strategies for Single-Pass Index Construction in Text Retrieval Systems. UW-TR-CS-2005-32. Technical report, University of Waterloo, Canada (October 2005)Google Scholar
  3. 3.
    Clarke, C., Craswell, N., Soboroff, I.: The TREC Terabyte Retrieval Track. SIGIR Forum 39(1), 25 (2005)CrossRefGoogle Scholar
  4. 4.
    Cutting, D.R., Pedersen, J.O.: Optimization for Dynamic Inverted Index Maintenance. In: Proceedings of the 13th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1990), pp. 405–411. ACM Press, New York (1990)CrossRefGoogle Scholar
  5. 5.
    Heinz, S., Zobel, J.: Efficient Single-Pass Index Construction for Text Databases. Journal of the American Society for Information Science and Technology 54(8), 713–729 (2003)CrossRefGoogle Scholar
  6. 6.
    Lester, N., Moffat, A., Zobel, J.: Fast On-Line Index Construction by Geometric Partitioning. In: Proceedings of the 14th ACM Conference on Information and Knowledge Management (CIKM 2005), Bremen, Germany (2005)Google Scholar
  7. 7.
    Lester, N., Zobel, J., Williams, H.E.: In-Place versus Re-Build versus Re- Merge: Index Maintenance Strategies for Text Retrieval Systems. In: CRPIT 26: Proceedings of the 27th Conference on Australasian Computer Science, pp. 15–23. Australian Computer Society, Inc. (2004)Google Scholar
  8. 8.
    Moffat, A.: Economical Inversion of Large Text Files. Computing Systems 5(2), 125–139 (1992)Google Scholar
  9. 9.
    Moffat, A., Bell, T.C.: In-Situ Generation of Compressed Inverted Files. Journal of the American Society of Information Science 46(7), 537–550 (1995)CrossRefGoogle Scholar
  10. 10.
    Shieh, W.-Y., Chung, C.-P.: A Statistics-Based Approach to Incrementally Update Inverted Files. Inf. Processing and Management 41(2), 275–288 (2005)CrossRefGoogle Scholar
  11. 11.
    Shoens, K.A., Tomasic, A., Garc´ıa-Molina, H.: SyntheticWorkload Performance Analysis of Incremental Updates. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1994), pp. 329–338 (1994)Google Scholar
  12. 12.
    Tomasic, A., García-Molina, H., Shoens, K.: Incremental Updates of Inverted Lists for Text Document Retrieval. In: Proceedings of the 1994 ACM SIGMOD Conference, pp. 289–300. ACM Press, New York (1994)CrossRefGoogle Scholar
  13. 13.
    Zobel, J., Heinz, S., Williams, H.E.: In-Memory Hash Tables for Accumulating Text Vocabularies. Information Processing Letters 80(6) (2001)Google Scholar
  14. 14.
    Zobel, J., Moffat, A., Ramamohanarao, K.: Inverted Files versus Signature Files for Text Indexing. ACM Trans. on Database Systems 23(4), 453–490 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Stefan Büttcher
    • 1
  • Charles L. A. Clarke
    • 1
  1. 1.School of Computer ScienceUniversity of WaterlooCanada

Personalised recommendations