Predicting Search Task Difficulty

  • Jaime Arguello
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8416)


Search task difficulty refers to a user’s assessment about the amount of effort required to complete a search task. Our goal in this work is to learn predictive models of search task difficulty. We evaluate features derived from the user’s interaction with the search engine as well as features derived from the user’s level of interest in the task and level of prior knowledge in the task domain. In addition to user-interaction features used in prior work, we evaluate features generated from scroll and mouse-movement events on the SERP. In some situations, we may prefer a system that can predict search task difficulty early in the search session. To this end, we evaluate features in terms of whole-session evidence and first-round evidence, which excludes all interactions starting with the second query. Our results found that the most predictive features were different for whole-session vs. first-round prediction, that mouseover features were effective for first-round prediction, and that level of interest and prior knowledge features did not improve performance.


Search Task Average Precision Search Behavior Mouse Movement Search Session 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, L.W., Krathwohl, D.R.: A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (2001)Google Scholar
  2. 2.
    Aula, A., Khan, R.M., Guan, Z.: How does search behavior change as search becomes more difficult? In: CHI, pp. 35–44 (2010)Google Scholar
  3. 3.
    Bell, D.J., Ruthven, I.: Searchers’ assessments of task complexity for web searching. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 57–71. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Byström, K., Järvelin, K.: Task complexity affects information seeking and use. Inf. Process. Manage. 31(2), 191–213 (1995)CrossRefGoogle Scholar
  5. 5.
    Campbell, D.J.: Task complexity: A review and analysis. The Academy of Management Review 13(1), 40–52 (1988)Google Scholar
  6. 6.
    Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: Liblinear: A library for large linear classification. JMLR 9, 1871–1874 (2008)zbMATHGoogle Scholar
  7. 7.
    Feild, H.A., Allan, J., Jones, R.: Predicting searcher frustration. In: SIGIR, pp. 34–41 (2010)Google Scholar
  8. 8.
    Jansen, B.J., Booth, D., Smith, B.: Using the taxonomy of cognitive learning to model online searching. Inf. Process. Manage. 45(6), 643–663 (2009)CrossRefGoogle Scholar
  9. 9.
    Kim, J.: Task difficulty as a predictor and indicator of web searching interaction. In: CHI, pp. 959–964 (2006)Google Scholar
  10. 10.
    Li, Y., Belkin, N.J.: A faceted approach to conceptualizing tasks in information seeking. Inf. Process. Manage. 44(6), 1822–1837 (2008)CrossRefGoogle Scholar
  11. 11.
    Liu, J., Cole, M.J., Liu, C., Bierig, R., Gwizdka, J., Belkin, N.J., Zhang, J., Zhang, X.: Search behaviors in different task types. In: JCDL, pp. 69–78 (2010)Google Scholar
  12. 12.
    Liu, J., Gwizdka, J., Liu, C., Belkin, N.J.: Predicting task difficulty for different task types. ASIS&T, 16:1–16:10 (2010)Google Scholar
  13. 13.
    Liu, J., Liu, C., Cole, M., Belkin, N.J., Zhang, X.: Exploring and predicting search task difficulty. In: CIKM, pp. 1313–1322 (2012)Google Scholar
  14. 14.
    Liu, J., Liu, C., Gwizdka, J., Belkin, N.J.: Can search systems detect users‘ task difficulty?: some behavioral signals. In: SIGIR, pp. 845–846 (2010)Google Scholar
  15. 15.
    Smucker, M.D., Allan, J., Carterette, B.: A comparison of statistical significance tests for information retrieval evaluation. In: CIKM, pp. 623–632 (2007)Google Scholar
  16. 16.
    White, R.W., Dumais, S.T.: Characterizing and predicting search engine switching behavior. In: CIKM, pp. 87–96 (2009)Google Scholar
  17. 17.
    Wood, R.E.: Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes 37(1), 60–82 (1986)CrossRefGoogle Scholar
  18. 18.
    Wu, W.-C., Kelly, D., Edwards, A., Arguello, J.: Grannies, tanning beds, tattoos and nascar: evaluation of search tasks with varying levels of cognitive complexity. In: IIIX, pp. 254–257 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jaime Arguello
    • 1
  1. 1.University of North Carolina at Chapel HillUSA

Personalised recommendations