Book Search Experiments: Investigating IR Methods for the Indexing and Retrieval of Books

  • Hengzhi Wu
  • Gabriella Kazai
  • Michael Taylor
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4956)


Through mass-digitization projects and with the use of OCR technologies, digitized books are becoming available on the Web and in digital libraries. The unprecedented scale of these efforts, the unique characteristics of the digitized material as well as the unexplored possibilities of user interactions make full-text book search an exciting area of information retrieval (IR) research. Emerging research questions include: How appropriate and effective are traditional IR models when applied to books? What book specific features (e.g., back-of-book index) should receive special attention during the indexing and retrieval processes? How can we tackle scalability? In order to answer such questions, we developed an experimental platform to facilitate rapid prototyping of a book search system as well as to support large-scale tests. Using this system, we performed experiments on a collection of 10 000 books, evaluating the efficiency of a novel multi-field inverted index and the effectiveness of the BM25F retrieval model adapted to books, using book-specific fields.


Book search multi-field indexing BM25F efficiency effectiveness 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Büttcher, S., Clarke, C.L.A., Lushman, B.: Hybrid index maintenance for growing text collections. In: Efthimiadis, E.N., Dumais, S.T., Hawking, D., Järvelin, K. (eds.) SIGIR, pp. 356–363. ACM, New York (2006)Google Scholar
  2. 2.
    Clark, J., DeRose, S.: XML Path Language (XPath) version 1.0. W3C Recommendation. Technical Report REC-xpath-19991116, W3C (World Wide Web Consortium) (November 1999),
  3. 3.
    Craswell, N., Zaragoza, H., Robertson, S.: Microsoft cambridge at TREC-14: Enterprise trackGoogle Scholar
  4. 4.
    Hawking, D.: Challenges in enterprise search. In: Schewe, K.-D., Williams, H.E. (eds.) ADC. CRPIT, vol. 27, pp. 15–24. Australian Computer Society (2004)Google Scholar
  5. 5.
    Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (ACM TOIS) 20(4), 422–446 (2002)CrossRefGoogle Scholar
  6. 6.
    Kaushik, R., Krishnamurthy, R., Naughton, J.F., Ramakrishnan, R.: On the integration of structure indexes and inverted lists. In: ICDE, p. 829 (2004)Google Scholar
  7. 7.
    Liu, S., Zou, Q., Chu, W.W.: Configurable indexing and ranking for XML information retrieval. In: Sanderson, M., Järvelin, K., Allan, J., Bruza, P. (eds.) SIGIR, pp. 88–95. ACM, New York (2004)Google Scholar
  8. 8.
    Moffat, A., Zobel, J.: Self-indexing inverted files for fast text retrieval. ACM Trans. Inf. Syst. 14(4), 349–379 (1996)CrossRefGoogle Scholar
  9. 9.
    Reddy, R., StClair, G.: The million book digital library projectGoogle Scholar
  10. 10.
    Robertson, S.E., Walker, S.: Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In: Croft, W.B., van Rijsbergen, C.J. (eds.) SIGIR, pp. 232–241. ACM/Springer (1994)Google Scholar
  11. 11.
    Robertson, S.E., Zaragoza, H., Taylor, M.J.: Simple BM25 extension to multiple weighted fields. In: Grossman, D., Gravano, L., Zhai, C., Herzog, O., Evans, D.A. (eds.) CIKM, pp. 42–49. ACM, New York (2004)Google Scholar
  12. 12.
    Sabine, Sabine: How people use books. Library Quarterly 56(4), 399–408 (1986)CrossRefGoogle Scholar
  13. 13.
    Taylor, M.J., Guiver, J., Robertson, S., Minka, T.: Softrank: Optimising non-smooth ranking metrics. In: First ACM International Conference on Web Search and Data Mining (WSDM), Stanford, California (2008)Google Scholar
  14. 14.
    Voorhees, E.M., Harman, D.K.: TREC: Experiment and Evaluation in Information Retrieval. MIT Press, Cambridge (2005)Google Scholar
  15. 15.
    Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Comput. Surv. 38(2) (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Hengzhi Wu
    • 1
  • Gabriella Kazai
    • 2
  • Michael Taylor
    • 2
  1. 1.Department of Computer ScienceQueen Mary,University of LondonUK
  2. 2.Microsoft ResearchCambridgeUK

Personalised recommendations