Abstract
We investigate using topic prediction data, as a summary of document content, to compute measures of search result quality. Unlike existing quality measures such as query clarity that require the entire content of the top-ranked results, class-based statistics can be computed efficiently online, because class information is compact enough to precompute and store in the index. In an empirical study we compare the performance of class-based statistics to their language-model counterparts for two performance-related tasks: predicting query difficulty and expansion risk. Our findings suggest that using class predictions can offer comparable performance to full language models while reducing computation overhead.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Amati, G., Carpineto, C., Romano, G.: Query difficulty, robustness, and selective application of query expansion. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 127–137. Springer, Heidelberg (2004)
Aslam, J.A., Pavlu, V.: Query hardness estimation using Jensen-Shannon divergence among multiple scoring functions. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 198–209. Springer, Heidelberg (2007)
Billerbeck, B.: Efficient Query Expansion. PhD thesis, RMIT University, Melbourne, Australia (2005)
Chickering, D., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. In: UAI 1997, pp. 80–89 (1997)
Chickering, D.M.: The WinMine toolkit. Technical Report MSR-TR-2002-103, Microsoft, Redmond, WA (2002)
Collins-Thompson, K., Callan, J.: Estimation and use of uncertainty in pseudo-relevance feedback. In: Proceedings of SIGIR 2007, pp. 303–310 (2007)
Cronen-Townsend, S., Croft, W.: Quantifying query ambiguity. In: Proceedings of HCL 2002, pp. 94–98 (2002)
Diaz, F.: Performance prediction using spatial autocorrelation. In: Proceedings of SIGIR 2007, pp. 583–590 (2007)
Hauff, C., Murdock, V., Baeza-Yates, R.: Improved query difficulty prediction for the web. In: Proceedings of CIKM 2008, pp. 439–448 (2008)
He, B., Ounis, I.: Query performance prediction. Information Systems 31, 585–594 (2006)
Heckerman, D., Chickering, D., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. Journal of Machine Learning Research 1, 49–75 (2000)
Lavrenko, V.: A Generative Theory of Relevance. PhD thesis, University of Massachusetts, Amherst (2004)
Lemur. Lemur toolkit for language modeling & retrieval (2002), http://www.lemurproject.org
Metzler, D., Croft, W.B.: Latent concept expansion using Markov Random Fields. In: Proceedings of SIGIR 2007, pp. 311–318 (2007)
Netscape Communication Corp. Open directory project, http://www.dmoz.org
Qiu, G., Liu, K., Bu, J., Chen, C., Kang, Z.: Quantify query ambiguity using ODP metadata. In: Proceedings of SIGIR 2007, pp. 697–698 (2007)
Smeaton, A., van Rijsbergen, C.J.: The retrieval effects of query expansion on a feedback document retrieval system. The Computer Journal 26(3), 239–246 (1983)
Song, R., Luo, Z., Wen, J.-R., Yu, Y., Hon, H.-W.: Identifying ambiguous queries in web search. In: Proceedings of WWW 2007, pp. 1169–1170 (2007)
Strohman, T., Metzler, D., Turtle, H., Croft, W.B.: Indri: A language model-based search engine for complex queries. In: Proceedings of the International Conference on Intelligence Analysis (2004)
Vinay, V., Cox, I.J., Milic-Frayling, N., Wood, K.: On ranking the effectiveness of searches. In: Proceedings of SIGIR 2005, pp. 398–404 (2005)
Winaver, M., Kurland, O., Domshlak, C.: Towards robust query expansion: model selection in the language modeling framework. In: Proceedings of SIGIR 2007, pp. 729–730 (2007)
YomTov, E., Fine, S., Carmel, D., Darlow, A.: Learning to estimate query difficulty. In: Proceedings of SIGIR 2005, pp. 512–519 (2005)
Zhou, Y., Croft, W.B.: Ranking robustness: a novel framework to predict query performance. In: Proceedings of CIKM 2006, pp. 567–574 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Collins-Thompson, K., Bennett, P.N. (2010). Predicting Query Performance via Classification. In: Gurrin, C., et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-12275-0_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12274-3
Online ISBN: 978-3-642-12275-0
eBook Packages: Computer ScienceComputer Science (R0)