Navigating the User Query Space

  • Ronan Cummins
  • Mounia Lalmas
  • Colm O’Riordan
  • Joemon M. Jose
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7024)


Query performance prediction (QPP) aims to automatically estimate the performance of a query. Recently there have been many attempts to use these predictors to estimate whether a perturbed version of a query will outperform the original version. In essence, these approaches attempt to navigate the space of queries in a guided manner.

In this paper, we perform an analysis of the query space over a substantial number of queries and show that (1) users tend to be able to extract queries that perform in the top 5% of all possible user queries for a specific topic, (2) that post-retrieval predictors outperform pre-retrieval predictors at the high end of the query space. And, finally (3), we show that some post retrieval predictors are better able to select high performing queries from a group of user queries for the same topic.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balasubramanian, N., Kumaran, G., Carvalho, V.R.: Exploring reductions for long web queries. In: SIGIR, pp. 571–578 (2010)Google Scholar
  2. 2.
    Carmel, D., Yom-Tov, E.: Estimating the Query Difficulty for Information Retrieval, 1st edn. Morgan and Claypool Publishers, San Francisco (2010)zbMATHGoogle Scholar
  3. 3.
    Cronen-Townsend, S., Zhou, Y., Bruce Croft, W.: Predicting query performance. In: SIGIR, pp. 299–306 (2002)Google Scholar
  4. 4.
    Cummins, R., Lalmas, M., Jose, J.: The limits of retrieval effectiveness. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 277–282. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Cummins, R., O’Riordan, C., Jose, J.: Improved query performance prediction using standard deviation. In: SIGIR 2011, ACM, New York (2011)Google Scholar
  6. 6.
    Hauff, C., Hiemstra, D., de Jong, F.: A survey of pre-retrieval query performance predictors. In: CIKM 2008, pp. 1419–1420. ACM, New York (2008)Google Scholar
  7. 7.
    He, B., Ounis, I.: Inferring query performance using pre-retrieval predictors. In: Apostolico, A., Melucci, M. (eds.) SPIRE 2004. LNCS, vol. 3246, pp. 43–54. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Kumaran, G., Allan, J.: Selective user interaction. In: CIKM 2007, pp. 923–926. ACM, New York (2007)Google Scholar
  9. 9.
    Kumaran, G., Carvalho, V.R.: Reducing long queries using query quality predictors. In: SIGIR, pp. 564–571 (2009)Google Scholar
  10. 10.
    Pérez-Iglesias, J., Araujo, L.: Standard deviation as a query hardness estimator. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 207–212. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Shtok, A., Kurland, O., Carmel, D.: Predicting query performance by query-drift estimation. In: Azzopardi, L., Kazai, G., Robertson, S., Rüger, S., Shokouhi, M., Song, D., Yilmaz, E. (eds.) ICTIR 2009. LNCS, vol. 5766, pp. 305–312. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Zhao, Y., Scholer, F., Tsegay, Y.: Effective pre-retrieval query performance prediction using similarity and variability evidence. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol. 4956, pp. 52–64. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ronan Cummins
    • 1
  • Mounia Lalmas
    • 2
  • Colm O’Riordan
    • 3
  • Joemon M. Jose
    • 1
  1. 1.School of Computing ScienceUniversity of GlasgowUK
  2. 2.Yahoo! ResearchBarcelonaSpain
  3. 3.Dept. of Information TechnologyNational University of IrelandGalwayIreland

Personalised recommendations