Skip to main content
SpringerLink
Log in

Searching, Learning, and Subtopic Ordering: A Simulation-Based Analysis

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13185))

Included in the following conference series:

  • 2545 Accesses

Abstract

Complex search tasks—such as those from the Search as Learning (SAL) domain—often result in users developing an information need composed of several aspects. However, current models of searcher behaviour assume that individuals have an atomic need, regardless of the task. While these models generally work well for simpler informational needs, we argue that searcher models need to be developed further to allow for the decomposition of a complex search task into multiple aspects. As no searcher model yet exists that considers both aspects and the SAL domain, we propose, by augmenting the Complex Searcher Model (CSM), the Subtopic Aware Complex Searcher Model (SACSM)—modelling aspects as subtopics to the user’s need. We then instantiate several agents (i.e., simulated users), with different subtopic selection strategies, which can be considered as different prototypical learning strategies (e.g., should I deeply examine one subtopic at a time, or shallowly cover several subtopics?). Finally, we report on the first large-scale simulated analysis of user behaviours in the SAL domain. Results demonstrate that the SACSM, under certain conditions, simulates user behaviours accurately.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In this paper, we refer to a model as a model of user behaviour.

  2. 2.

    This research has been supported by NWO projects SearchX (639.022.722) and Aspasia (015.013.027).

  3. 3.

    We consider a search session as interactions with a search interface, which can include the issuing of multiple queries—and the examination of multiple documents.

  4. 4.

    Agents are simulated users that are able to make judgements as to the relevancy/attractiveness of information without recourse to relevance information [29].

  5. 5.

    Refer to Sect. 4.3 for more information on the use of Wikipedia articles.

  6. 6.

    https://github.com/ArthurCamara/CHIIR21-SAL-Scaffolding/blob/master/data/blocklist.txt (All URLs last accessed January 18\(^\mathrm{th}\), 2022.).

  7. 7.

    As noted in Sect. 3, we do not have explicit relevance judgements.

  8. 8.

    As an example, the following are extracted keywords for the topic Ethics: ethical, ontology, propositions, consequentialism, normative and principles.

  9. 9.

    This number was decided experimentally, as it showed to be the best to distinguish between the different methods trialled.

References

  1. Albers, M., Kim, L.: Information design for the small-screen interface: an overview of web design issues for personal digital assistants. Tech. Comm. 49(1), 45–60 (2002)

    Google Scholar 

  2. Azzopardi, L.: The economics in interactive information retrieval. In: Proceedings of 34\(^{\rm th}\) ACM SIGIR, pp. 15–24 (2011)

    Google Scholar 

  3. Azzopardi, L.: Modelling interaction with economic models of search. In: Proceedings of 37\(^{\rm th}\) ACM SIGIR, pp. 3–12 (2014)

    Google Scholar 

  4. Azzopardi, L., Järvelin, K., Kamps, J., Smucker, M.D.: Report on the SIGIR 2010 workshop on the simulation of interaction. SIGIR Forum 44(2), 35–47 (2011)

    Article  Google Scholar 

  5. Azzopardi, L., Zuccon, G.: Two scrolls or one click: a cost model for browsing search results. In: Proceedings of 38\(^{\rm th}\) ECIR, pp. 696–702 (2016)

    Google Scholar 

  6. Baskaya, F., Keskustalo, H., Järvelin, K.: Modeling behavioral factors in interactive information retrieval. In: Proceedings of 22\(^{\rm nd}\) ACM CIKM, pp. 2297–2302 (2013)

    Google Scholar 

  7. Bates, M.J.: The design of browsing and berrypicking techniques for the online search interface. Online Review (1989)

    Google Scholar 

  8. Belkin, N.J.: The cognitive viewpoint in information science. J. Info. Sci. 16(1), 11–15 (1990)

    Article  Google Scholar 

  9. Bhattacharya, N., Gwizdka, J.: Relating eye-tracking measures with changes in knowledge on search tasks. In: Proceedings of 13\(^{\rm th}\) ACM ETRA, pp. 1–5 (2018)

    Google Scholar 

  10. Câmara, A., Roy, N., Maxwell, D., Hauff, C.: Searching to learn with instructional scaffolding. In: Proceedings of 6\(^{\rm th}\) ACM CHIIR, pp. 209–218 (2021)

    Google Scholar 

  11. Carterette, B., Kanoulas, E., Yilmaz, E.: Simulating simple user behavior for system effectiveness evaluation. In: Proceedings of the 20\(^{\rm th}\) ACM CIKM, pp. 611–620 (2011)

    Google Scholar 

  12. Crescenzi, A., Kelly, D., Azzopardi, L.: Time pressure and system delays in information search. In: Proceedings of 38\(^{\rm th}\) ACM SIGIR, pp. 767–770 (2015)

    Google Scholar 

  13. Dai, W., Srihari, R.: Minimal document set retrieval. In: Proceedings of 14\(^{\rm th}\) ACM CIKM, pp. 752–759 (2005)

    Google Scholar 

  14. Dietz, L., Verma, M., Radlinski, F., Craswell, N.: TREC complex answer retrieval overview. TREC (2017)

    Google Scholar 

  15. Eickhoff, C., Teevan, J., White, R., Dumais, S.: Lessons from the journey: a query log analysis of within-session learning. In: Proceedings of 7\(^{\rm th}\) ACM WSDM, pp. 223–232 (2014)

    Google Scholar 

  16. Ellis, D.: Modeling the information-seeking patterns of academic researchers: a grounded theory approach. Libr. Q. 63(4), 469–486 (1993)

    Article  Google Scholar 

  17. Fuhr, N.: A probability ranking principle for interactive information retrieval. Inf. Retrieval 11(3), 251–265 (2008)

    Article  Google Scholar 

  18. Hearst, M.: Search User Interfaces. Cambridge University Press (2009)

    Google Scholar 

  19. Hearst, M., Plaunt, C.: Subtopic structuring for full-length document access. In: Proceedings of 16\(^{\rm th}\) ACM SIGIR, pp. 59–68 (1993)

    Google Scholar 

  20. Ingwersen, P., Järvelin, K.: The Turn: Integration of Information Seeking and Retrieval in Context, vol. 18, p. 448. Springer, Dordrecht (2006). https://doi.org/10.1007/1-4020-3851-8

  21. Jiang, Z., Wen, J.R., Dou, Z., Zhao, W.X., Nie, J.Y., Yue, M.: Learning to diversify search results via subtopic attention. In: Proceedings of 40\(^{\rm th}\) ACM SIGIR, pp. 545–554 (2017)

    Google Scholar 

  22. Kashyap, A., Hristidis, V., Petropoulos, M.: FACeTOR: cost-driven exploration of faceted query results. In: Proceedings of 19\(^{\rm th}\) ACM CIKM, pp. 719–728 (2010)

    Google Scholar 

  23. Kuhlthau, C.C.: Developing a model of the library search process: cognitive and affective aspects, pp. 232–242 (1988)

    Google Scholar 

  24. Kuperman, V., Stadthagen-Gonzalez, H., Brysbaert, M.: Age-of-acquisition ratings for 30,000 English words. Behav. Res. Methods 44(4), 978–990 (2012)

    Article  Google Scholar 

  25. Liu, C., Song, X.: How do information source selection strategies influence users’ learning outcomes. In: Proceedings of 3\(^{\rm th}\) ACM CHIIR, pp. 257–260 (2018)

    Google Scholar 

  26. Marchionini, G.: Exploratory search: from finding to understanding. Commun. ACM 49(4), 41–46 (2006)

    Article  Google Scholar 

  27. Maxwell, D.: Modelling Search and Stopping in Interactive Information Retrieval. Ph.D. thesis, University of Glasgow, Scotland (2019)

    Google Scholar 

  28. Maxwell, D., Azzopardi, L.: Stuck in traffic: how temporal delays affect search behaviour. In: Proceedings of 5\(^{\rm th}\) IIiX, pp. 155–164 (2014)

    Google Scholar 

  29. Maxwell, D., Azzopardi, L.: Agents, simulated users and humans: an analysis of performance and behaviour. In: CIKM, pp. 731–740. ACM (2016)

    Google Scholar 

  30. Maxwell, D., Azzopardi, L.: Simulating interactive information retrieval: SimIIR: a framework for the simulation of interaction. In: Proceedings of 39\(^{\rm th}\) ACM SIGIR, pp. 1141–1144 (2016)

    Google Scholar 

  31. Maxwell, D., Azzopardi, L.: Information scent, searching and stopping. In: Proceedings of 40\(^{\rm th}\) ECIR, pp. 210–222 (2018)

    Google Scholar 

  32. Maxwell, D., Azzopardi, L., Järvelin, K., Keskustalo, H.: An initial investigation into fixed and adaptive stopping strategies. In: Proceedings of 38\(^{\rm th}\) ACM SIGIR, pp. 903–906 (2015)

    Google Scholar 

  33. Maxwell, D., Azzopardi, L., Järvelin, K., Keskustalo, H.: Searching and stopping: an analysis of stopping rules and strategies. In: Proceedings of 24\(^{\rm th}\) ACM CIKM, pp. 313–322 (2015)

    Google Scholar 

  34. Meij, E., Weerkamp, W., de Rijke, M.: A query model based on normalized log-likelihood. In: Proceedings of 18\(^{\rm th}\) ACM CIKM, pp. 1903–1906 (2009)

    Google Scholar 

  35. Moffat, A., Scholer, F., Thomas, P.: Models and metrics: IR evaluation as a user process. In: Proceedings of 17\(^{\rm th}\) ADCS, pp. 47–54 (2012)

    Google Scholar 

  36. Moraes, F., Putra, S.R., Hauff, C.: Contrasting search as a learning activity with instructor-designed learning. In: Proceedings of 27\(^{\rm th}\) ACM CIKM, pp. 167–176 (2018)

    Google Scholar 

  37. Navalpakkam, V., Jentzsch, L., Sayres, R., Ravi, S., Ahmed, A., Smola, A.: Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts. In: Proceedings of the 22\(^{\rm nd}\) WWW, pp. 953–964 (2013)

    Google Scholar 

  38. Nguyen, T.N., Kanhabua, N.: Leveraging dynamic query subtopics for time-aware search result diversification. In: Proceedings of 36\(^{\rm th}\) ECIR, pp. 222–234 (2014)

    Google Scholar 

  39. O’Brien, H.L., Kampen, A., Cole, A.W., Brennan, K.: The role of domain knowledge in search as learning. In: Proceedings of 5\(^{\rm th}\) ACM CHIIR, pp. 313–317 (2020)

    Google Scholar 

  40. Ong, K., Järvelin, K., Sanderson, M., Scholer, F.: Qwerty: the effects of typing on web search behavior. In: Proceedings of 3\(^{\rm rd}\) ACM CHIIR, pp. 281–284 (2018)

    Google Scholar 

  41. Pardi, G., von Hoyer, J., Holtz, P., Kammerer, Y.: The role of cognitive abilities and time spent on texts and videos in a multimodal searching as learning task. In: Proceedings of 5\(^{\rm th}\) ACM CHIIR, pp. 378–382 (2020)

    Google Scholar 

  42. Peytchev, A., Couper, M.P., McCabe, S.E., Crawford, S.D.: Web survey design: paging versus scrolling. Intl. J. Pub. Opin. Q. 70(4), 596–607 (2006)

    Article  Google Scholar 

  43. Pirolli, P., Card, S.: Information foraging. Psychol. Rev. 106(4), 643 (1999)

    Article  Google Scholar 

  44. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. arXiv preprint arXiv:1908.10084 (2019)

  45. Roy, N., Câmara, A., Maxwell, D., Hauff, C.: Incorporating widget positioning in interaction models of search behaviour. In: Proceedings of 11\(^{\rm th}\) ACM ICTIR, pp. 53–62 (2021)

    Google Scholar 

  46. Roy, N., Moraes, F., Hauff, C.: Exploring users’ learning gains within search sessions. In: Proceedings of 5\(^{\rm th}\) ACM CHIIR, pp. 432–436 (2020)

    Google Scholar 

  47. Roy, N., Torre, M.V., Gadiraju, U., Maxwell, D., Hauff, C.: Note the highlight: incorporating active reading tools in a search as learning environment. In: Proceedings of 6\(^{\rm th}\) ACM CHIIR, pp. 229–238 (2021)

    Google Scholar 

  48. Schurman, E., Brutlag, J.: Performance related changes and their user impact. In: O’Reilly Velocity Conference (2009)

    Google Scholar 

  49. Syed, R., Collins-Thompson, K.: Retrieval algorithms optimized for human learning. In: Proceedings of 40\(^{\rm th}\) ACM SIGIR, pp. 555–564 (2017)

    Google Scholar 

  50. Syed, R., Collins-Thompson, K.: Exploring document retrieval features associated with improved short-and long-term vocabulary learning outcomes. In: Proceedings of 3\(^{\rm th}\) ACM CHIIR, pp. 191–200 (2018)

    Google Scholar 

  51. Takaki, T., Fujii, A., Ishikawa, T.: Associative document retrieval by query subtopic analysis and its application to invalidity patent search. In: Proceedings of 13\(^{\rm th}\) ACM CIKM, pp. 399–405 (2004)

    Google Scholar 

  52. Thomas, P., Moffat, A., Bailey, P., Scholer, F.: Modeling decision points in user search behavior. In: Proceedings of 5\(^{\rm th}\) IIiX, pp. 239–242 (2014)

    Google Scholar 

  53. Wang, J., Zhu, J.: Portfolio theory of information retrieval. In: Proceedings of 32\(^{\rm nd}\) ACM SIGIR, pp. 115–122 (2009)

    Google Scholar 

  54. Wildemuth, B.M.: The effects of domain knowledge on search tactic formulation. J. Am. Soc. Info. Sci. Tech. 55(3), 246–258 (2004)

    Google Scholar 

  55. Wilson, M.J., Wilson, M.L.: A comparison of techniques for measuring sensemaking and learning within participant-generated summaries. J. Am. Soc. Info. Sci. Tech. 64(2), 291–306 (2013)

    Google Scholar 

  56. Wu, W.C., Kelly, D., Sud, A.: Using information scent and need for cognition to understand online search behavior. In: Proceedings of 37\(^{\rm th}\) ACM SIGIR, pp. 557–566 (2014)

    Google Scholar 

  57. Zhai, C., Cohen, W.W., Lafferty, J.: Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. SIGIR Forum 49(1), 2–9 (2015)

    Article  Google Scholar 

  58. Zuccon, G., Azzopardi, L., Hauff, C., van Rijsbergen, C.K.: Estimating interference in the QPRP for subtopic retrieval. In: Proceedings of 33\(^{\rm rd}\) ACM SIGIR, pp. 741–742 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Delft University of Technology, Delft, The Netherlands

    Arthur Câmara, David Maxwell & Claudia Hauff

Authors
  1. Arthur Câmara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Maxwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claudia Hauff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arthur Câmara .

Editor information

Editors and Affiliations

  1. Martin Luther University Halle-Wittenberg, Halle, Germany

    Matthias Hagen

  2. Leiden University, Leiden, The Netherlands

    Suzan Verberne

  3. University of Glasgow, Glasgow, UK

    Craig Macdonald

  4. University of Duisburg-Essen, Essen, Germany

    Christin Seifert

  5. University of Stavanger, Stavanger, Norway

    Krisztian Balog

  6. Norwegian University of Science and Technology, Trondheim, Norway

    Kjetil Nørvåg

  7. University of Stavanger, Stavanger, Norway

    Vinay Setty

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Câmara, A., Maxwell, D., Hauff, C. (2022). Searching, Learning, and Subtopic Ordering: A Simulation-Based Analysis. In: Hagen, M., et al. Advances in Information Retrieval. ECIR 2022. Lecture Notes in Computer Science, vol 13185. Springer, Cham. https://doi.org/10.1007/978-3-030-99736-6_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-99736-6_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-99735-9

  • Online ISBN: 978-3-030-99736-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation