Advertisement

On the impact of group size on collaborative search effectiveness

  • Felipe Moraes
  • Kilian Grashoff
  • Claudia HauffEmail author
Article
  • 39 Downloads

Abstract

While today’s web search engines are designed for single-user search, over the years research efforts have shown that complex information needs—which are explorative, open-ended and multi-faceted—can be answered more efficiently and effectively when searching in collaboration. Collaborative search (and sensemaking) research has investigated techniques, algorithms and interface affordances to gain insights and improve the collaborative search process. It is not hard to imagine that the size of the group collaborating on a search task significantly influences the group’s behaviour and search effectiveness. However, a common denominator across almost all existing studies is a fixed group size—usually either pairs, triads or in a few cases four users collaborating. Investigations into larger group sizes and the impact of group size dynamics on users’ behaviour and search metrics have so far rarely been considered—and when, then only in a simulation setup. In this work, we investigate in a large-scale user experiment to what extent those simulation results carry over to the real world. To this end, we extended our collaborative search framework SearchX with algorithmic mediation features and ran a large-scale experiment with more than 300 crowd-workers. We consider the collaboration group size as a dependent variable, and investigate collaborations between groups of up to six people. We find that most prior simulation-based results on the impact of collaboration group size on behaviour and search effectiveness cannot be reproduced in our user experiment.

Keywords

Collaborative search Search effectiveness Interactive search 

Notes

Acknowledgements

This research has been supported by NWO projects LACrOSSE (612.001.605) and SearchX (639.022.722).

References

  1. Amershi, S., & Morris, M. R. (2008). Cosearch: A system for co-located collaborative web search. In CHI’08 (pp. 1647–1656).Google Scholar
  2. Azzopardi, L. (2014). Modelling interaction with economic models of search. In SIGIR’14.Google Scholar
  3. Azzopardi, L., Kelly, D., & Brennan, K. (2013). How query cost affects search behavior. In SIGIR’13 (pp. 23–32).Google Scholar
  4. Bailey, P., Chen, L., Grosenick, S., Jiang, L., Li, Y., Reinholdtsen, P., Salada, C., Wang, H., & Wong, S. (2012). User task understanding: A web search engine perspective. In NII shonan meeting on whole-session evaluation of interactive information retrieval systems, Kanagawa, Japan.Google Scholar
  5. Böhm, T., Klas, C.-P., & Hemmje, M. (2016). Towards a probabilistic model for supporting collaborative information access. Information Retrieval Journal, 19(5), 487–509.CrossRefGoogle Scholar
  6. Brennan, S. E., Chen, X., Dickinson, C. A., Neider, M. B., & Zelinsky, G. J. (2008). Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 106(3), 1465–1477.CrossRefGoogle Scholar
  7. Capra, R., Chen, A. T., Hawthorne, K., Arguello, J., Shaw, L., & Marchionini, G. (2012). Design and evaluation of a system to support collaborative search. Proceedings of the Association for Information Science and Technology, 49(1), 1–10.Google Scholar
  8. Collins-Thompson, K., Hansen, P., & Hauff, C. (2017). Search as learning (Dagstuhl Seminar 17092). Dagstuhl Reports, 7(2), 135–162. ISSN 2192-5283.  https://doi.org/10.4230/DagRep.7.2.135. http://drops.dagstuhl.de/opus/volltexte/2017/7357.
  9. Diriye, A., & Golovchinsky, G. (2012). Querium: A session-based collaborative search system. In ECIR’12 (pp. 583–584).Google Scholar
  10. Foley, C., & Smeaton, A. F. (2010). Division of labour and sharing of knowledge for synchronous collaborative information retrieval. IPM, 46(6), 762–772.Google Scholar
  11. Gao, Y., Reddy, M., & Jansen, B. J. (2016). Shop together, search together: Collaborative e-commerce. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems (pp. 2081–2087). ACM.Google Scholar
  12. Golovchinsky, G., Pickens, J., & Back, M. (2009). A taxonomy of collaboration in online information seeking. arXiv preprint arXiv:0908.0704.
  13. González-Ibáñez, R., & Shah, C. (2011). Coagmento: A system for supporting collaborative information seeking. ASIST, 48(1), 1–4.Google Scholar
  14. González-Ibáñez, R., Haseki, M., & Shah, C. (2013). Let’s search together, but not too close! An analysis of communication and performance in collaborative information seeking. IPM, 49(5), 1165–1179.Google Scholar
  15. Hansen, P., & Järvelin, K. (2005). Collaborative information retrieval in an information-intensive domain. IPM, 41(5), 1101–1119.Google Scholar
  16. Htun, N. N., Halvey, M., & Baillie, L. (2015). Towards quantifying the impact of non-uniform information access in collaborative information retrieval. In SIGIR’15, (pp. 843–846).Google Scholar
  17. Htun, N. N., Halvey, M., & Baillie, L. (2017). How can we better support users with non-uniform information access in collaborative information retrieval? In CHIIR’17, (pp. 235–244).Google Scholar
  18. Joho, H., Hannah, D., & Jose, J. M. (2008). Comparing collaborative and independent search in a recall-oriented task. In IIiX’08 (pp. 89–96).Google Scholar
  19. Joho, H., Hannah, D., & Jose, J. M. (2009). Revisiting IR techniques for collaborative search strategies. In ECIR’09 (pp. 66–77).Google Scholar
  20. Kapetanios, E. (2008). Quo vadis computer science: From turing to personal computer, personal content and collective intelligence. Data & Knowledge Engineering, 67(2), 286–292.CrossRefGoogle Scholar
  21. Kelly, R., & Payne, S. J. (2014). Collaborative web search in context: A study of tool use in everyday tasks. In CSCW’14 (pp. 807–819).Google Scholar
  22. Kules, B., & Shneiderman, B. (2008). Users can change their web search tactics: Design guidelines for categorized overviews. Information Processing and Management, 44(2), 463–484.CrossRefGoogle Scholar
  23. Lavrenko, V., & Croft, W. B. (2001). Relevance based language models. In SIGIR’01 (pp. 120–127).Google Scholar
  24. Manku, G. S., Jain, A., & Das Sarma, A. (2007). Detecting near-duplicates for web crawling. In WWW’07, (pp. 141–150).Google Scholar
  25. Morris, M. R. (2007). Collaborating alone and together: Investigating persistent and multi-user web search activities. In SIGIR’07, (pp. 23–27).Google Scholar
  26. Morris, M. R. (2008). A survey of collaborative web search practices. In CHI’08 (pp. 1657–1660).Google Scholar
  27. Morris, M. R. (2013). Collaborative search revisited. In CSCW’13 (pp. 1181–1192).Google Scholar
  28. Morris, M. R., & Horvitz, E. (2007). Searchtogether: An interface for collaborative web search. In UIST’07 (pp. 3–12).Google Scholar
  29. Morris, M. R., Teevan, J., & Bush, S. (2008). Enhancing collaborative web search with personalization: Groupization, smart splitting, and group hit-highlighting. In CSCW’08 (pp. 481–484).Google Scholar
  30. Paul, S. A., & Morris, M. R. (2009). Cosense: Enhancing sensemaking for collaborative web search. In CHI’09, (pp. 1771–1780).Google Scholar
  31. Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163.CrossRefGoogle Scholar
  32. Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., & Back, M. (2008). Algorithmic mediation for collaborative exploratory search. In SIGIR’08, (pp. 315–322).Google Scholar
  33. Putra, S. R., Moraes, F., & Hauff, C. (2018). Searchx: Empowering collaborative search research. In ACM SIGIR.Google Scholar
  34. Shah, C. (2010). Collaborative information seeking: A literature review. In Advances in librarianship, (pp. 3–33). Emerald Group Publishing Limited.Google Scholar
  35. Shah, C., & González-Ibáñez, R. (2011). Evaluating the synergic effect of collaboration in information seeking. In SIGIR’11 (pp. 913–922). ACM.Google Scholar
  36. Shah, C., Pickens, J., & Golovchinsky, G. (2010). Role-based results redistribution for collaborative information retrieval. IPM, 46(6), 773–781.Google Scholar
  37. Smeaton, A. F., Lee, H., Foley, C., & McGivney, S. (2007). Collaborative video searching on a tabletop. Multimedia Systems, 12(4–5), 375–391.CrossRefGoogle Scholar
  38. Smyth, B., Freyne, J., Coyle, M., Briggs, P., & Balfe, E. (2004). I-SPY—anonymous, community-based personalization by collaborative meta-search. In Research and development in intelligent systems XX, (pp. 367–380). Berlin: SpringerGoogle Scholar
  39. Soulier, L., Shah, C., & Tamine, L. (2014a). User-driven system-mediated collaborative information retrieval. In SIGIR’14 (pp. 485–494).Google Scholar
  40. Soulier, L., Tamine, L., & Bahsoun, W. (2014b). On domain expertise-based roles in collaborative information retrieval. IPM, 50(5), 752–774.Google Scholar
  41. Tamine, L., & Soulier, L. (2015). Understanding the impact of the role factor in collaborative information retrieval. In CIKM’15 (pp. 43–52).Google Scholar
  42. Teevan, J., Morris, M. R., & Bush, S. (2009). Discovering and using groups to improve personalized search. In WSDM’09 (pp. 15–24).Google Scholar
  43. Twidale, M. B., Nichols, D. M., & Paice, C. D. (1997). Browsing is a collaborative process. IPM, 33(6), 761–783.Google Scholar
  44. Voorhees, E. M. (2006). The TREC 2005 robust track. In ACM SIGIR Forum (Vol. 40, pp. 41–48). ACM.Google Scholar
  45. Zhai, C., & Lafferty, J. (2004). A study of smoothing methods for language models applied to information retrieval. TOIS, 22(2), 179–214.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Delft University of TechnologyDelftThe Netherlands

Personalised recommendations