Query Processing in Highly-Loaded Search Engines

  • Daniele Broccolo
  • Craig Macdonald
  • Salvatore Orlando
  • Iadh Ounis
  • Raffaele Perego
  • Fabrizio Silvestri
  • Nicola Tonellotto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8214)

Abstract

While Web search engines are built to cope with a large number of queries, query traffic can exceed the maximum query rate supported by the underlying computing infrastructure. We study how response times and results vary when, in presence of high loads, some queries are either interrupted after a fixed time threshold elapses or dropped completely. Moreover, we introduce a novel dropping strategy, based on machine learned performance predictors to select the queries to drop in order to sustain the largest possible query rate with a relative degradation in effectiveness.

Keywords

Distributed Search Engines Efficiency Effectiveness Throughput 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Anh, V.N., de Kretser, O., Moffat, A.: Vector-space ranking with effective early termination. In: Proceedings of SIGIR, pp. 35–42 (2001)Google Scholar
  2. 2.
    Barroso, L.A., Dean, J., Holzle, U.: Web search for a planet: The Google cluster architecture. IEEE Micro 23(2), 22–28 (2003)CrossRefGoogle Scholar
  3. 3.
    Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.: Efficient query evaluation using a two-level retrieval process. In: Proceedings of CIKM, pp. 426–434 (2003)Google Scholar
  4. 4.
    Moffat, A., Zobel, J.: Self-indexing inverted files for fast text retrieval. ACM Trans. Inf. Syst. 14(4), 349–379 (1996)CrossRefGoogle Scholar
  5. 5.
    Tonellotto, N., Macdonald, C., Ounis, I.: Efficient and Effective Retrieval using Selective Pruning. In: Proceedings of WSDM (2013)Google Scholar
  6. 6.
    Macdonald, C., Tonellotto, N., Ounis, I.: Learning to Predict Response Times for Online Query Scheduling. In: Proceedings of SIGIR, pp. 621–630 (2012)Google Scholar
  7. 7.
    Carterette, B., Pavlu, V., Fang, H., Kanoulas, E.: Million Query Track 2009 Overview. In: Proceedings of TREC (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Daniele Broccolo
    • 1
    • 2
  • Craig Macdonald
    • 3
  • Salvatore Orlando
    • 1
    • 2
  • Iadh Ounis
    • 3
  • Raffaele Perego
    • 2
  • Fabrizio Silvestri
    • 2
  • Nicola Tonellotto
    • 2
  1. 1.Università Ca’Foscari of VeniceItaly
  2. 2.ISTI-CNR of PisaItaly
  3. 3.University of GlasgowUK

Personalised recommendations