Advertisement

Information Scent, Searching and Stopping

Modelling SERP Level Stopping Behaviour
  • David Maxwell
  • Leif Azzopardi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)

Abstract

Current models and measures of the Interactive Information Retrieval (IIR) process typically assume that a searcher will always examine the first snippet in a given Search Engine Results Page (SERP), and then with some probability or cutoff, he or she will stop examining snippets and/or documents in the ranked list (snippet level stopping). Prior work has however shown that searchers will form an initial impression of the SERP, and will often abandon a page without clicking on or inspecting in detail any snippets or documents. That is, the information scent affects their decision to continue. In this work, we examine whether considering the information scent of a page leads to better predictions of stopping behaviour. In a simulated analysis, grounded with data from a prior user study, we show that introducing a SERP level stopping strategy can improve the performance attained by simulated users, resulting in an increase in gain across most snippet level stopping strategies. When compared to actual search and stopping behaviour, incorporating SERP level stopping offers a closer approximation than without. These findings show that models and measures that naïvely assume snippets and documents in a ranked list are actually examined in detail are less accurate, and that modelling SERP level stopping is required to create more realistic models of the search process.

Notes

Acknowledgements

Our thanks to Horaţiu Bota and Alastair Maxwell for their feedback – including Horaţiu’s helpful comments on our results. We would also like to thank the anonymous reviewers for their comments and feedback. Finally, the lead author is funded by the UK Government though the EPSRC, grant number 1367507.

References

  1. 1.
    Bailey, P., Moffat, A., Scholer, F., Thomas, P.: User variability and IR system evaluation. In: Proceedings of 38th ACM SIGIR, pp. 625–634 (2015)Google Scholar
  2. 2.
    Baskaya, F., Keskustalo, H., Järvelin, K.: Modeling behavioral factors in interactive information retrieval. In: Proceedings of 22nd ACM CIKM, pp. 2297–2302 (2013)Google Scholar
  3. 3.
    Browne, G., Pitts, M., Wetherbe, J.: Stopping rule use during web-based search. In: Proceedings of HICSS-38, p. 271b (2005)Google Scholar
  4. 4.
    Card, S., Pirolli, P., Van Der Wege, M., Morrison, J., Reeder, R., Schraedley, P., Boshart, J.: Information scent as a driver of web behavior graphs: results of a protocol analysis method for web usability. In: Proceedings of 19th ACM CHI, pp. 498–505 (2001)Google Scholar
  5. 5.
    Chi, E., Pirolli, P., Chen, K., Pitkow, J.: Using information scent to model user information needs and actions and the web. In: Proceedings of 19th ACM CHI, pp. 490–497 (2001)Google Scholar
  6. 6.
    Cooper, W.: On selecting a measure of retrieval effectiveness part II. Implementation of the philosophy. J. Am. Soc. Info. Sci. 24(6), 413–424 (1973)CrossRefGoogle Scholar
  7. 7.
    Diriye, A., White, R., Buscher, G., Dumais, S.: Leaving so soon? Understanding and predicting web search abandonment rationales. In: Proceedings of 21st ACM CIKM, pp. 1025–1034 (2012)Google Scholar
  8. 8.
    Hagen, M., Michel, M., Stein, B.: Simulating ideal and average users. In: Proceedings of 12th AIRS, pp. 138–154 (2016)Google Scholar
  9. 9.
    Hassan, A., White, R.: Personalized models of search satisfaction. In: Proceedings of 22nd ACM CIKM, pp. 2009–2018 (2013)Google Scholar
  10. 10.
    Hassan, A., Shi, X., Craswell, N., Ramsey, B.: Beyond clicks: query reformulation as a predictor of search satisfaction. In: Proceedings of 22nd CIKM, pp. 2019–2028 (2013)Google Scholar
  11. 11.
    Hearst, M.: Search user interfaces. Cambridge University Press, Cambridge (2009)Google Scholar
  12. 12.
    Ingwersen, P., Järvelin, K.: The Turn: Integration of Information Seeking and Retrieval in Context. Springer, Dordrecht (2005).  https://doi.org/10.1007/1-4020-3851-8 zbMATHGoogle Scholar
  13. 13.
    Keskustalo, H., Järvelin, K., Pirkola, A., Sharma, T., Lykke, M.: Test collection-based IR evaluation needs extension toward sessions – a case of extremely short queries. In: Proceedings of 5th AIRS, pp. 63–74 (2009)Google Scholar
  14. 14.
    Kiseleva, J., Kamps, J., Nikulin, V., Makarov, N.: Behavioral dynamics from the SERP’s perspective: what are failed SERPs and how to fix them? In: Proceedings of 24th ACM CIKM, pp. 1561–1570 (2015)Google Scholar
  15. 15.
    Kraft, D., Lee, T.: Stopping rules and their effect on expected search length. IPM 15(1), 47–58 (1979)zbMATHGoogle Scholar
  16. 16.
    Loumakis, F., Stumpf, S., Grayson, D.: This image smells good: effects of image information scent in search engine results pages. In: Proceedings of 20th ACM CIKM, pp. 475–484 (2011)Google Scholar
  17. 17.
    Marshall, C.C., Shipman III, F.M.: Spatial hypertext and the practice of information triage. In: Proceedings of 8th ACM HYPERTEXT, pp. 124–133 (1997)Google Scholar
  18. 18.
    Maxwell, D., Azzopardi, L.: Agents, simulated users and humans: an analysis of performance and behaviour. In: Proceedings of 25th ACM CIKM, pp. 731–740 (2016)Google Scholar
  19. 19.
    Maxwell, D., Azzopardi, L., Järvelin, K., Keskustalo, H.: An initial investigation into fixed and adaptive stopping strategies. In: Proceedings of 38th ACM SIGIR, pp. 903–906 (2015)Google Scholar
  20. 20.
    Maxwell, D., Azzopardi, L., Järvelin, K., Keskustalo, H.: Searching and stopping: an analysis of stopping rules and strategies. In: Proceedings of 24th ACM CIKM, pp. 313–322 (2015)Google Scholar
  21. 21.
    Maxwell, D., Azzopardi, L., Moshfeghi, Y.: A study of snippet length and informativeness: behaviour, performance and UX. In: Proceedings of 40th ACM SIGIR (2017)Google Scholar
  22. 22.
    McNair, J.N.: Optimal giving-up times and the marginal value theorem. Am. Nat. 119(4), 511–529 (1982)CrossRefGoogle Scholar
  23. 23.
    Moffat, A., Bailey, P., Scholer, F., Thomas, P.: INST: An adaptive metric for IR evaluation. In: Proceedings of 20th ADCS, pp. 5:1–5:4 (2015)Google Scholar
  24. 24.
    Moffat, A., Zobel, J.: Rank-biased precision for measurement of retrieval effectiveness. ACM Trans. Info. Syst. 27(1), 2:1–2:27 (2008)CrossRefGoogle Scholar
  25. 25.
    Nickles, K.: Judgment-based and reasoning-based stopping rules in decision making under uncertainty. Ph.D. thesis, University of Minnesota (1995)Google Scholar
  26. 26.
    Olston, C., Chi, E.: ScentTrails: integrating browsing and searching on the web. ACM Trans. Comput. Hum. Interact. 10(3), 177–197 (2003)CrossRefGoogle Scholar
  27. 27.
    Ong, K., Järvelin, K., Sanderson, M., Scholer, F.: Using information scent to understand mobile and desktop web search behavior. In: Proceedings of 40th ACM SIGIR (2017)Google Scholar
  28. 28.
    Pirolli, P.: Information Foraging Theory: Adaptive Interaction with Information, 1st edn. Oxford University Press, New York (2007)CrossRefGoogle Scholar
  29. 29.
    Pirolli, P., Card, S.: Informayion foraging in information access environments. In: Proceedings of 13th ACM SIGCHI, pp. 51–58 (1995)Google Scholar
  30. 30.
    Pirolli, P., Card, S.: Information foraging. Psychol. Rev. 106, 643–675 (1999)CrossRefGoogle Scholar
  31. 31.
    Smucker, M., Clarke, C.: Modeling optimal switching behavior. In: Proceedings of 1st ACM CHIIR, pp. 317–320 (2016)Google Scholar
  32. 32.
    Stephens, D., Krebs, J.: Foraging Theory. Princeton University Press, Princeton (1986)Google Scholar
  33. 33.
    Sundar, S., Knobloch-Westerwick, S., Hastall, M.: News cues: information scent and cognitive heuristics. J. Am. Soc. Inf. Sci. Technol. 58(3), 366–378 (2007)CrossRefGoogle Scholar
  34. 34.
    Thomas, P., Moffat, A., Bailey, P., Scholer, F.: Modeling decision points in user search behavior. In: Proceedings of 5th IIiX, pp. 239–242 (2014)Google Scholar
  35. 35.
    Tombros, A., Sanderson, M.: Advantages of query biased summaries in information retrieval. In: Proceedings of 21st ACM SIGIR, pp. 2–10 (1998)Google Scholar
  36. 36.
    Wu, W.: How far will you go? Using need for closure and information scent to model search stopping behavior. In: Proceedings of 4th IIiX, pp. 328–328 (2012)Google Scholar
  37. 37.
    Wu, W., Kelly, D., Sud, A.: Using information scent and need for cognition to understand online search behavior. In: Proceedings of 37th ACM SIGIR, pp. 557–566 (2014)Google Scholar
  38. 38.
    Zach, L.: When is “enough” enough? Modeling the information-seeking and stopping behavior of senior arts administrators. J. Am. Soc. Info. Sci. Tech. 56(1), 23–35 (2005)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computing ScienceUniversity of GlasgowGlasgowScotland
  2. 2.Computer and Information SciencesUniversity of StrathclydeGlasgowScotland

Personalised recommendations