Using Coherence-Based Measures to Predict Query Difficulty

  • Jiyin He
  • Martha Larson
  • Maarten de Rijke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4956)


We investigate the potential of coherence-based scores to predict query difficulty. The coherence of a document set associated with each query word is used to capture the quality of a query topic aspect. A simple query coherence score, QC-1, is proposed that requires the average coherence contribution of individual query terms to be high. Two further query scores, QC-2 and QC-3, are developed by constraining QC-1 in order to capture the semantic similarity among query topic aspects. All three query coherence scores show the correlation with average precision necessary to make them good predictors of query difficulty. Simple and efficient, the measures require no training data and are competitive with language model-based clarity scores.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amati, G., Carpineto, C., Romano, G.: Query difficulty, robustness and selective application of query expansion. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 127–137. Springer, Heidelberg (2004)Google Scholar
  2. 2.
    Cronen-Townsend, S., Croft, W.B.: Quantifying query ambiguity. In: HLT 2002, pp. 94–98 (2002)Google Scholar
  3. 3.
    Cronen-Townsend, S., Zhou, Y., Croft, W.B.: Predicting query performance. In: SIGIR 2002, pp. 299–306 (2002)Google Scholar
  4. 4.
    Harman, D., Buckley, C.: The NRRC reliable information access (RIA) workshop. In: SIGIR 2004, pp. 528–529 (2004)Google Scholar
  5. 5.
    He, B., Ounis, I.: Query performance prediction. Inf. Syst. 31(7), 585–594 (2006)CrossRefGoogle Scholar
  6. 6.
    Pilpel, Y., Sudarsanam, P., Church, G.M.: Identifying regulatory networks by combinatiorial analysis of promoter elements. Nat. Genet. 29, 153–159 (2001)CrossRefGoogle Scholar
  7. 7.
    Voorhees, E.M.: The TREC robust retrieval track. SIGIR Forum 39, 11–20 (2005)CrossRefGoogle Scholar
  8. 8.
    Yom-Tov, E., Fine, S., Carmel, D., Darlow, A.: Learning to estimate query difficulty. In: SIGIR 2005, pp. 512–519 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jiyin He
    • 1
  • Martha Larson
    • 1
  • Maarten de Rijke
    • 1
  1. 1.ISLA, University of Amsterdam 

Personalised recommendations