Highlighting Weasel Sentences for Promoting Critical Information Seeking on the Web

  • Fumiaki SaitoEmail author
  • Yoshiyuki Shoji
  • Yusuke YamamotoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11881)


This paper proposes a system that highlights weasel sentences while browsing webpages. The term weasel sentence is defined in the context of this paper as a quotation with an unknown or unidentifiable source. Following this definition, the system automatically detects weasel sentences in browsed webpages. Then, we investigate how highlighting weasel sentences affects the search behaviors and decision making of the users searching for information on the web. An online user study yielded the following results: (1) Highlighting the weasel sentences encouraged participants to invest more time in web browsing and to view a larger number of webpages. (2) The effect of (1) was more significant when participants were familiar with the search topics. (3) Web browsing elicited less change in the confidence of the search answers when participants were familiar with the given topics. The findings provide insights into how users can avoid gathering misleading on the web.


Web browsing Information credibility Critical information seeking Human factor User interface 



This work was supported in part by Grants-in-Aid for Scientific Research (18H03243, 18H03244, 18H03494, 18KT0097, 18K18161, 16H02906) from MEXT of Japan.


  1. 1.
    American Library Association, Association for College and Research Libraries: Information Literacy Competency Standards for Higher Education. Technical report (2000)Google Scholar
  2. 2.
    Bateman, S., Teevan, J., White, R.W.: The search dashboard: how reflection and comparison impact search behavior. In: Proceedings of CHI (2012)Google Scholar
  3. 3.
    Bürkner, P.C.: brms: an R package for Bayesian multilevel models using Stan. J. Stat. Softw. 80(1), 1–8 (2017)CrossRefGoogle Scholar
  4. 4.
    Dong, X.L., et al.: Knowledge-based trust: estimating the trustworthiness of Web sources. In: Proceedings of the VLDB Endowment, vol. 8, no. 9, pp. 938–949 (2015)CrossRefGoogle Scholar
  5. 5.
    Ennals, R., Trushkowsky, B., Agosta, J.M.: Highlighting disputed claims on the Web. In: Proceedings of WWW (2010)Google Scholar
  6. 6.
    Fogg, B.J., Tseng, H.: The elements of computer credibility. In: Proceedings of CHI (1999)Google Scholar
  7. 7.
    Hasan Dalip, D., André Gonçalves, M., Cristo, M., Calado, P.: Automatic quality assessment of content created collaboratively by Web communities: a case study of wikipedia. In: Proceedings of JCDL (2009)Google Scholar
  8. 8.
    Hofmann, K., Mitra, B., Radlinski, F., Shokouhi, M.: An eye-tracking study of user interactions with query auto completion. In: Proceedings of CIKM (2014)Google Scholar
  9. 9.
    Ieong, S., Mishra, N., Sadikov, E., Zhang, L.: Domain bias in Web search. In: Proceedings of WSDM (2012)Google Scholar
  10. 10.
    Kahneman, D.: Thinking, Fast and Slow. Macmillan, London (2011)Google Scholar
  11. 11.
    Kay, M., Nelson, G.L., Hekler, E.B.: Researcher-centered design of statistics: why Bayesian statistics better fit the culture and incentives of HCI. In: Proceedings of CHI (2016Google Scholar
  12. 12.
    Kruschke, J.: Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press, Cambridge (2014)zbMATHGoogle Scholar
  13. 13.
    Lee, J., Walker, E., Burleson, W., Kay, M., Buman, M., Hekler, E.B.: Self-experimentation for behavior change: design and formative evaluation of two approaches. In: Proceedings of CHI (2017)Google Scholar
  14. 14.
    Leong, C.W., Cucerzan, S.: Supporting factual statements with evidence from the Web. In: Proceedings of CIKM (2012)Google Scholar
  15. 15.
    Liao, Q.V., Fu, W.T.: Expert voices in echo chambers: effects of source expertise indicators on exposure to diverse opinions. In: Proceedings of CHI (2014Google Scholar
  16. 16.
    Liu, C., White, R.W., Dumais, S.: Understanding web browsing behaviors through Weibull analysis of dwell time. In: Proceedings of SIGIR (2010)Google Scholar
  17. 17.
    Metzger, M.J.: Making sense of credibility on the Web: models for evaluating online information and recommendations for future research. J. Am. Soc. Inf. Sci. Technol. 58(13), 2078–2091 (2007)CrossRefGoogle Scholar
  18. 18.
    Nakamura, S., et al.: Trustworthiness analysis of Web search results. In: Proceedings of ECDL (2007)Google Scholar
  19. 19.
    Pan, B., Hembrooke, H., Joachims, T., Lorigo, L., Gay, G., Granka, L.: In Google we trust: users’ decisions on rank, position, and relevance. J. Comput. Mediated Commun. 12(3), 801–823 (2007)CrossRefGoogle Scholar
  20. 20.
    Schwarz, J., Morris, M.: Augmenting Web pages and search results to support credibility assessment. In: Proceedings of CHI (2011)Google Scholar
  21. 21.
    Wang, W.Y.: Liar, liar pants on fire: a new benchmark dataset for fake news detection. In: Proceedings of ACL (2017)Google Scholar
  22. 22.
    White, R.: Beliefs and biases in web search. In: Proceedings of SIGIR (2013)Google Scholar
  23. 23.
    Yamamoto, T., Yamamoto, Y., Fujita, S.: Exploring people’s attitudes and behaviors toward careful information seeking in Web search. In: Proceedings of CIKM (2018)Google Scholar
  24. 24.
    Yamamoto, Y., Yamamoto, T.: Query priming for promoting critical thinking in Web search. In: Proceedings of CHIIR (2018)Google Scholar
  25. 25.
    Yamamoto, Y., Yamamoto, T., Ohshima, H., Kawakami, H.: Web access literacy scale to evaluate how critically users can browse and search for Web information. In: Proceedings of WebSci (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Shizuoka UniversityHamamatsuJapan
  2. 2.Aoyama Gakuin UniversitySagamihara-shiJapan

Personalised recommendations