Running Time Prediction for Web Search Queries
Large scale Web search engines have to process thousands of queries per second and each query has to be solved within a fraction of a second. To achieve this goal, search engines rely on sophisticated services capable of processing large amounts of data. One of these services is the search service (or index service) which is in charge of computing the top-k document results for user queries. Predicting in advance the response time of queries has practical applications in efficient administration of hardware resources assigned to query processing. In this paper, we propose and evaluate a query running time prediction algorithm that is based on a discrete Fourier transform which models the index as a collection of signals to obtain patterns. Results show that our approach performs at least as effectively as well-known prediction algorithms in the literature, while significantly improving computational efficiency.
KeywordsWAND Inverted files Multi-threading
This research was partially funded by Basal funds FB0001, Conicyt, Chile; PMI USA 1204 and PICT 2014-1146.
- 1.Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.Y.: Efficient query evaluation using a two-level retrieval process. In: CIKM, pp. 426–434 (2003)Google Scholar
- 2.Macdonald, N.T.C., Ounis, I.: Learning to predict response times for online query scheduling. In: SIGIR, pp. 621–630 (2012)Google Scholar
- 3.Chakrabarti, K., Chaudhuri, S., Ganti, V.: Interval-based pruning for top-k processing over compressed lists. In: ICDE, pp. 709–720 (2011)Google Scholar
- 4.Cronen-Townsend, S., Zhou, Y., Croft, W.B.: Predicting query performance. In: SIGIR, pp. 299–306 (2002)Google Scholar
- 5.Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In: SIGIR, pp. 993–1002 (2011)Google Scholar
- 6.Kim, S., He, Y., Hwang, S., Elnikety, S., Choi, S.: Delayed-dynamic-selective (DDS) prediction for reducing extreme tail latency in web search. In: WSDM, pp. 7–16 (2015)Google Scholar
- 7.Park, L., Ramamohanarao, K., Palaniswami, M.: Fourier domain scoring: a novel document ranking method. TKDE 16(5), 529–539 (2004)Google Scholar
- 9.Tonellotto, N., Macdonald, C., Ounis, I.: Efficient and effective retrieval using selective pruning. In: WSDM, pp. 63–72 (2013)Google Scholar