Investigating exploratory search activities based on the stratagem level in digital libraries

Article

Abstract

In this paper, we present the results of a user study on exploratory search activities in a social science digital library. We conducted a user study with 32 participants with a social sciences background—16 postdoctoral researchers and 16 students—who were asked to solve a task on searching related work to a given topic. The exploratory search task was performed in a 10-min time slot. The use of certain search activities is measured and compared to gaze data recorded with an eye tracking device. We use a novel tree graph representation to visualise the users’ search patterns and introduce a way to combine multiple search session trees. The tree graph representation is capable of creating one single tree for multiple users and identifying common search patterns. In addition, the information behaviour of students and postdoctoral researchers is being compared. The results show that search activities on the stratagem level are frequently utilised by both user groups. The most heavily used search activities were keyword search, followed by browsing through references and citations, and author searching. The eye tracking results showed an intense examination of documents metadata, especially on the level of citations and references. When comparing the group of students and postdoctoral researchers, we found significant differences regarding gaze data on the area of the journal name of the seed document. In general, we found a tendency of the postdoctoral researchers to examine the metadata records more intensively with regard to dwell time and the number of fixations. By creating combined session trees and deriving subtrees from those, we were able to identify common patterns like economic (explorative) and exhaustive (navigational) behaviour. Our results show that participants utilised multiple search strategies starting from the seed document, which means that they examined different paths to find related publications.

Keywords

Search process Stratagems Interactive information retrieval Information behaviour Digital libraries Eye tracking Session tree Social sciences 

References

  1. 1.
    Athukorala, K., Hoggan, E., Lehtiö, A., Ruotsalo, T., Jacucci, G.: Information-seeking behaviors of computer scientists: challenges for electronic literature search tools. Proc. Am. Soc. Inf. Sci. Technol. 50(1), 1–11 (2013)CrossRefGoogle Scholar
  2. 2.
    Athukorala, K., Gowacka, D., Jacucci, G., Oulasvirta, A., Vreeken, J.: Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks. J. Assoc. Inf. Sci. Technol. 67(11), 2635–2651 (2016). doi:10.1002/asi.23617 CrossRefGoogle Scholar
  3. 3.
    Aula, A., Majaranta, P., Räihä, K.J.: Eye-tracking reveals the personal styles for search result evaluation. In: IFIP Conference on Human–Computer Interaction. Springer, pp 1058–1061 (2005)Google Scholar
  4. 4.
    Bates, M.J.: Information search tactics. JASIS 30(4), 205–214 (1979)CrossRefGoogle Scholar
  5. 5.
    Bates, M.J.: The design of browsing and berrypicking techniques for the online search interface. Online Rev. 13(5), 407–424 (1989)CrossRefGoogle Scholar
  6. 6.
    Bates, M.J.: Where should the person stop and the information search interface start? Inf. Process. Manag. 26(5), 575–591 (1990)CrossRefGoogle Scholar
  7. 7.
    Bierig, R., Gwizdka, J., Cole, M.J.: A user-centered experiment and logging framework for interactive information retrieval. In: Proceedings of the SIGIR 2009 Workshop on Understanding the User: Logging and Interpreting User Interactions in Information Search and Retrieval, pp. 8–11 (2009)Google Scholar
  8. 8.
    Brajnik, G., Mizzaro, S., Tasso, C., Venuti, F.: Strategic help in user interfaces for information retrieval. JASIST 53(5), 343–358 (2002). doi:10.1002/asi.10035 CrossRefGoogle Scholar
  9. 9.
    Buscher, G., Dengel, A., Biedert, R., Elst, L.V.: Attentive documents: eye tracking as implicit feedback for information retrieval and beyond. ACM Trans. Interact. Intell. Syst. (TiiS) 1(2), 9 (2012a)Google Scholar
  10. 10.
    Buscher, G., White, R.W., Dumais, S., Huang, J.: Large-scale analysis of individual and task differences in search result page examination strategies. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, WSDM ’12. ACM, New York, NY, pp. 373–382 (2012b). doi:10.1145/2124295.2124341
  11. 11.
    Carevic, Z., Mayr, P.: Extending search facilities via bibliometric-enhanced stratagems. In: Proceedings of the 2nd Workshop on Bibliometric-Enhanced Information Retrieval, CEUR-WS.org, Vienna, Austria, pp. 40–46 (2015)Google Scholar
  12. 12.
    Carevic, Z., Mayr, P.: Survey on high-level search activities based on the stratagem level in digital libraries. In: 20th International Conference on Theory and Practice of Digital Libraries (TPDL 2016) (2016). doi:10.1007/978-3-319-43997-6_5
  13. 13.
    Cole, M.J., Hendahewa, C., Belkin, N.J., Shah, C.: User activity patterns during information search. ACM Trans. Inf. Syst. (TOIS) 33(1), 1 (2015)CrossRefGoogle Scholar
  14. 14.
    Cutrell, E., Guan, Z.: What are you looking for? An eye-tracking study of information usage in web search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 407–416 (2007)Google Scholar
  15. 15.
    Ellis, D.: A behavioural approach to information retrieval system design. J. Doc. 45(3), 171–212 (1989)CrossRefGoogle Scholar
  16. 16.
    Fields, B., Keith, S., Blandford, A.: Designing for expert information finding strategies. In: Fincher, S., Markopoulos, P., Moore, D., Ruddle, R. (eds.) People and Computers XVIII–Design for Life: Proceedings of HCI 2004, 89–102. Springer, London (2005). doi:10.1007/1-84628-062-1_6
  17. 17.
    Goldberg, J.H., Stimson, M.J., Lewenstein, M., Scott, N., Wichansky, A.M.: Eye tracking in web search tasks: design implications. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications. ACM, pp. 51–58 (2002)Google Scholar
  18. 18.
    Granka, L.A., Joachims, T., Gay, G.: Eye-tracking analysis of user behavior in www search. In: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, pp. 478–479 (2004)Google Scholar
  19. 19.
    Hienert, D., Sawitzki, F., Mayr, P.: Digital library research in action: supporting information retrieval in sowiport. D-Lib Mag. 21(3), 8 (2015). doi:10.1045/march2015-hienert Google Scholar
  20. 20.
    Joo, S., Xie, I.: How do users’ search tactic selections influence search outputs in exploratory search tasks? In: Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries. ACM, New York, NY, pp. 397–398 (2013). doi:10.1145/2467696.2467761
  21. 21.
    Kang, R., Fu, W.T.: Exploratory information search by domain experts and novices. In: Proceedings of the 15th International Conference on Intelligent User Interfaces. ACM, pp. 329–332 (2010)Google Scholar
  22. 22.
    Kelly, D.: Methods for evaluating interactive information retrieval systems with users. Found. Trends Inf. Retr. 3(12), 1–224 (2009)Google Scholar
  23. 23.
    Kriewel, S., Fuhr, N.: Adaptive search suggestions for digital libraries. In: Asian Digital Libraries: Looking Back 10 Years and Forging New Frontiers (ICADL 2007), pp. 220–229 (2007)Google Scholar
  24. 24.
    Kriewel, S., Fuhr, N.: An evaluation of an adaptive search suggestion system. In: 32nd European Conference on Information Retrieval Research (ECIR 2010). Springer, pp. 544–555 (2010)Google Scholar
  25. 25.
    Kules, B., Capra, R., Banta, M., Sierra, T.: What do exploratory searchers look at in a faceted search interface? In: Proceedings of the 9th ACM/IEEE-CS Joint Conference on Digital Libraries. ACM, pp. 313–322 (2009)Google Scholar
  26. 26.
    Loizides, F., Photiades, T., Mavri, A., Zaphiris, P.: On interactive interfaces for semi-structured academic document seeking and relevance decision making. New Rev. Inf. Netw. 19(2), 67–95 (2014)CrossRefGoogle Scholar
  27. 27.
    Lorigo, L., Haridasan, M., Brynjarsdóttir, H., Xia, L., Joachims, T., Gay, G., Granka, L., Pellacini, F., Pan, B.: Eye tracking and online search: lessons learned and challenges ahead. J. Am. Soc. Inform. Sci. Technol. 59(7), 1041–1052 (2008)CrossRefGoogle Scholar
  28. 28.
    Marchionini, G.: Exploratory search: from finding to understanding. Commun. ACM 49(4), 41–46 (2006)CrossRefGoogle Scholar
  29. 29.
    Mayr, P.: How do practitioners, Ph.D. students and postdocs in the social sciences assess topic-specific recommendations? In: Proceedings of the Joint Workshop on Bibliometric-Enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL2016), CEUR-WS.org, Newark, NJ, USA, pp. 84–92 (2016)Google Scholar
  30. 30.
    Meho, L.I., Tibbo, H.R.: Modeling the information-seeking behavior of social scientists: Ellis’s study revisited. J. Am. Soc. Inform. Sci. Technol. 54(6), 570–587 (2003)CrossRefGoogle Scholar
  31. 31.
    Mitsui, M., Shah, C., Belkin, N.J.: Extracting information seeking intentions for web search sessions. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, SIGIR ’16, pp. 841–844 (2016). doi:10.1145/2911451.2914746
  32. 32.
    Mutschke, P., Mayr, P.: Science models for search. A study on combining scholarly information retrieval and scientometrics. Scientometrics 102(3), 2323–2345 (2015). doi:10.1007/s11192-014-1485-2 CrossRefGoogle Scholar
  33. 33.
    O’Day, V.L., Jeffries, R.: Orienting in an information landscape: how information seekers get from here to there. In: Proceedings of the INTERCHI ’93. IOS Press, pp. 438–445 (1993)Google Scholar
  34. 34.
    Reichle, E.D., Pollatsek, A., Rayner, K.: Using EZ reader to simulate eye movements in nonreading tasks: a unified framework for understanding the eye-mind link. Psychol. Rev. 119(1), 155 (2012)CrossRefGoogle Scholar
  35. 35.
    Tran, V.T., Fuhr, N.: Quantitative analysis of search sessions enhanced by gaze tracking with dynamic areas of interest. In: The International Conference on Theory and Practice of Digital Libraries 2012. Springer, pp. 468–473 (2012a)Google Scholar
  36. 36.
    Tran, V.T., Fuhr, N.: Using eye-tracking with dynamic areas of interest for analyzing interactive information retrieval. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, pp. 1165–1166 (2012b)Google Scholar
  37. 37.
    White, R.W., Drucker, S.M.: Investigating behavioral variability in web search. In: Proceedings of the 16th International Conference on World Wide Web, WWW ’07. ACM, New York, NY, pp. 21–30 (2007). doi:10.1145/1242572.1242576
  38. 38.
    Wildemuth, B.M.: The effects of domain knowledge on search tactic formulation. J. Am. Soc. Inform. Sci. Technol. 55(3), 246–258 (2004)CrossRefGoogle Scholar
  39. 39.
    Wilson, M.L., Schraefel, M.: A longitudinal study of exploratory and keyword search. In: Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL ’08. ACM, New York, NY, pp. 52–56 (2008). doi:10.1145/1378889.1378899
  40. 40.
    Xie, H.I.: Patterns between interactive intentions and information-seeking strategies. Inf. Process. Manag. 38(1), 55–77 (2002). doi:10.1016/S0306-4573(01)00018-8 CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.GESIS - Leibniz Institute for the Social SciencesCologneGermany
  2. 2.RheinMain University of Applied SciencesWiesbadenGermany

Personalised recommendations