Abstract
Large search engines process thousands of queries per second over billions of documents, making a huge performance gap between disjunctive and conjunctive text queries in query processing. An important class of optimization techniques called top-k processing is therefore used to narrow this gap. In this paper, we present an improvement to the MaxScore optimization, which is the most efficient known document-at-a-time (DAAT) top-k processing method. Essentially, our approach can speed up MaxScore method by enabling skipping not just in non-essential lists as the original method does but also in essential lists, thus the name Essential List Skipping MaxScore (ELS-MaxScore), and providing more promising candidates for scoring. Experiments with TREC GOV2 collection show that our ELS-MaxScore processes significantly less elements, thus reduces the average query latency by almost 18% over the MaxScore baseline and 84% over the disjunctive DAAT baseline, while still returns the same results as the disjunctive evaluation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dean, J.: Challenges in building large-scale information retrieval systems: invited talk. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, p. 1. ACM (2009)
Puppin, D., Silvestri, F., Laforenza, D.: Query-driven document partitioning and collection selection. In: Proceedings of the 1st International Conference on Scalable Information Systems, p. 34. ACM (2006)
Melink, S., Raghavan, S., Yang, B., Garcia-Molina, H.: Building a distributed full-text index for the web. ACM Transactions on Information Systems (TOIS) 19(3), 217–241 (2001)
Yan, H., Ding, S., Suel, T.: Compressing term positions in web indexes. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 147–154. ACM (2009)
Broder, A.Z., Carmel, D., Herscovici, M., Soffer, A., Zien, J.: Efficient query evaluation using a two-level retrieval process. In: Proceedings of the Twelfth International Conference on Information and Knowledge Management, pp. 426–434. ACM (2003)
Chakrabarti, K., Chaudhuri, S., Ganti, V.: Interval-based pruning for top-k processing over compressed lists. In: 2011 IEEE 27th International Conference on Data Engineering (ICDE), pp. 709–720. IEEE (2011)
Turtle, H., Flood, J.: Query evaluation: strategies and optimizations. Information Processing & Management 31(6), 831–850 (1995)
Strohman, T., Turtle, H., Croft, W.B.: Optimization strategies for complex queries. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 219–225. ACM (2005)
Jonassen, S., Bratsberg, S.E.: Efficient compressed inverted index skipping for disjunctive text-queries. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 530–542. Springer, Heidelberg (2011)
Ding, S., Suel, T.: Faster top-k document retrieval using block-max indexes. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 993–1002. ACM (2011)
Dimopoulos, C., Nepomnyachiy, S., Suel, T.: Optimizing top-k document retrieval strategies for block-max indexes. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 113–122. ACM (2013)
Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Computing Surveys (CSUR)Â 38(2), 6 (2006)
Büttcher, S., Clarke, C., Cormack, G.V.: Information retrieval: Implementing and evaluating search engines. The MIT Press (2010)
Moffat, A., Zobel, J.: Self-indexing inverted files for fast text retrieval. ACM Transactions on Information Systems (TOIS) 14(4), 349–379 (1996)
Fontoura, M., Josifovski, V., Liu, J., Venkatesan, S., Zhu, X., Zien, J.: Evaluation strategies for top-k queries over memory-resident inverted indexes. Proceedings of the VLDB Endowment 4(12), 1213–1224 (2011)
Lacour, P., Macdonald, C., Ounis, I.: Efficiency comparison of document matching techniques. In: Proc. ECIR (2008)
Lester, N., Moffat, A., Webber, W., Zobel, J.: Space-limited ranked query evaluation using adaptive pruning. In: Ngu, A.H.H., Kitsuregawa, M., Neuhold, E.J., Chung, J.-Y., Sheng, Q.Z. (eds.) WISE 2005. LNCS, vol. 3806, pp. 470–477. Springer, Heidelberg (2005)
Strohman, T., Croft, W.B.: Efficient document retrieval in main memory. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 175–182. ACM (2007)
Chierichetti, F., Lattanzi, S., Mari, F., Panconesi, A.: On placing skips optimally in expectation. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 15–24. ACM (2008)
Boldi, P., Vigna, S.: Compressed perfect embedded skip lists for quick inverted-index lookups. In: Consens, M.P., Navarro, G. (eds.) SPIRE 2005. LNCS, vol. 3772, pp. 25–28. Springer, Heidelberg (2005)
Zukowski, M., Heman, S., Nes, N., Boncz, P.: Super-scalar ram-cpu cache compression. In: Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006, p. 59. IEEE (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Jiang, K., Yang, Y. (2014). Faster MaxScore Query Processing with Essential List Skipping. In: Luo, X., Yu, J.X., Li, Z. (eds) Advanced Data Mining and Applications. ADMA 2014. Lecture Notes in Computer Science(), vol 8933. Springer, Cham. https://doi.org/10.1007/978-3-319-14717-8_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-14717-8_43
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14716-1
Online ISBN: 978-3-319-14717-8
eBook Packages: Computer ScienceComputer Science (R0)