Abstract
To clarify and address the errors that occur in model parameter estimations and behavior predictions, researchers may need to start with investigating the hidden gaps between rational agent and human that are ignored or covered by oversimplified model assumptions. These gaps could occur in both factual, ad hoc retrieval and whole-session interactive retrieval and involve multiple aspects of search interactions, including not only user characteristics and their search strategies but also search task features, search interfaces, as well as situational factors. In this chapter, we summarize and briefly discuss the gaps we identified between simplified rational assumptions and empirically confirmed human biases and then propose a preliminary bias-aware evaluation framework to describe the connections between different stages of search sessions and diverse types of biases. The identified gaps will serve as the basis for developing bias-aware user models, search systems, and evaluation metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agosto, D. E. (2002). Bounded rationality and satisficing in young people’s Web-based decision making. Journal of the American society for Information Science and Technology, 53(1), 16–27. https://doi.org/10.1002/asi.10024
Asghar, H. M. (2015). Measuring information seeking through Facebook: Scale development and initial evidence of Information Seeking in Facebook Scale (ISFS). Computers in Human Behavior, 52, 259–270. https://doi.org/10.1016/j.chb.2015.06.005
Azzopardi, L. (2011). The economics in interactive information retrieval. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 15–24). ACM. https://doi.org/10.1145/2009916.2009923
Azzopardi, L. (2014). Modelling interaction with economic models of search. In Proceedings of the 37th ACM SIGIR Conference on Research & Development in Information Retrieval (pp. 3–12). ACM. https://doi.org/10.1145/2600428.2609574
Azzopardi, L. (2021). Cognitive biases in search: A review and reflection of cognitive biases in information retrieval. In Proceedings of the 2021 ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 27–37). ACM. https://doi.org/10.1145/3406522.3446023
Azzopardi, L., Mackenzie, J., & Moffat, A. (2021). ERR is not C/W/L: Exploring the relationship between expected reciprocal rank and other metrics. In Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval (pp. 231–237). ACM. https://doi.org/10.1145/3471158.3472239
Azzopardi, L., Thomas, P., & Craswell, N. (2018). Measuring the utility of search engine result pages: An information foraging based measure. In Proceedings of the 41st ACM SIGIR Conference on Research & Development in Information Retrieval (pp. 605–614). ACM. https://doi.org/10.1145/3209978.3210027
Azzopardi, L., & Zuccon, G. (2016). An analysis of the cost and benefit of search interactions. In Proceedings of the 2016 ACM SIGIR International Conference on the Theory of Information Retrieval (pp. 59–68). ACM. https://doi.org/10.1145/2970398.2970412
Barnes, J. H., Jr. (1984). Cognitive biases and their impact on strategic planning. Strategic Management Journal, 5(2), 129–137. https://doi.org/10.1002/smj.4250050204
Barnfield, M. (2020). Think twice before jumping on the bandwagon: Clarifying concepts in research on the bandwagon effect. Political Studies Review, 18(4), 553–574. https://doi.org/10.1177/1478929919870691
Brown, T., & Liu, J. (2022). A reference dependence approach to enhancing early prediction of session behavior and satisfaction. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (pp. 1–5). ACM. https://doi.org/10.1145/3529372.3533294
Chapelle, O., Metlzer, D., Zhang, Y., & Grinspan, P. (2009). Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (pp. 621–630). ACM. https://doi.org/10.1145/1645953.1646033
Charness, G., & Dave, C. (2017). Confirmation bias with motivated beliefs. Games and Economic Behavior, 104, 1–23. https://doi.org/10.1016/j.geb.2017.02.015
Chen, T. (2021). A systematic integrative review of cognitive biases in consumer health information seeking: Emerging perspective of behavioral information research. Journal of Documentation, 77(3), 798–823. https://doi.org/10.1108/JD-01-2020-0004
Chen, N., Zhang, F., & Sakai, T. (2022). Constructing better evaluation metrics by incorporating the anchoring effect into the user model. In Proceedings of the 45rd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM. https://doi.org/10.1145/3477495.3531953
Chen, Y., Zhou, K., Liu, Y., Zhang, M., & Ma, S. (2017). Meta-evaluation of online and offline web search evaluation metrics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 15–24). ACM. https://doi.org/10.1145/3077136.3080804
Chuklin, A., Markov, I., & Rijke, M. D. (2015). Click models for web search. Synthesis Lectures on Information concepts, Retrieval, and Services, 7(3), 1–115. https://doi.org/10.2200/S00654ED1V01Y201507ICR043
Clarke, C. L., Vtyurina, A., & Smucker, M. D. (2020). Offline evaluation without gain. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval (pp. 185–192). ACM. https://doi.org/10.1145/3409256.3409816
Cole, M., Liu, J., Belkin, N. J., Bierig, R., Gwizdka, J., Liu, C., Zhang, J., & Zhang, X. (2009). Usefulness as the criterion for evaluation of interactive information retrieval. In Proceedings of the Third Workshop on Human-Computer Interaction and Information Retrieval (pp. 1–4). HCIR.
Croskerry, P. (2003). The importance of cognitive errors in diagnosis and strategies to minimize them. Academic Medicine, 78(8), 775–780.
Diaz, F., Mitra, B., Ekstrand, M. D., Biega, A. J., & Carterette, B. (2020). Evaluating stochastic rankings with expected exposure. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp. 275–284). ACM. https://doi.org/10.1145/3340531.3411962
Eickhoff, C. (2018). Cognitive biases in crowdsourcing. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (pp. 162–170). ACM. https://doi.org/10.1145/3159652.3159654
Ekstrand, M. D., Burke, R., & Diaz, F. (2019). Fairness and discrimination in retrieval and recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1403–1404). ACM. https://doi.org/10.1145/3331184.3331380
Gäde, M., Koolen, M., Hall, M., Bogers, T., & Petras, V. (2021). A manifesto on resource re-use in interactive information retrieval. In Proceedings of the 2021 ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 141–149). ACM. https://doi.org/10.1145/3406522.3446056
Gao, R., & Shah, C. (2019). How fair can we go: Detecting the boundaries of fairness optimization in information retrieval. In Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval (pp. 229–236). ACM. https://doi.org/10.1145/3341981.3344215
Gatian, A. W. (1994). Is user satisfaction a valid measure of system effectiveness? Information & Management, 26(3), 119–131. https://doi.org/10.1016/0378-7206(94)90036-1
Gwizdka, J. (2010). Distribution of cognitive load in web search. Journal of the American Society for Information Science and Technology, 61(11), 2167–2187. https://doi.org/10.1002/asi.21385
Harman, D. (2011). Information retrieval evaluation. Synthesis Lectures on Information Concepts, Retrieval, and Services, 3(2), 1–119. https://doi.org/10.2200/S00368ED1V01Y201105ICR019
Hendahewa, C., & Shah, C. (2017). Evaluating user search trails in exploratory search tasks. Information Processing & Management, 53(4), 905–922. https://doi.org/10.1016/j.ipm.2017.04.001
Hofmann, K., Li, L., & Radlinski, F. (2016). Online evaluation for information retrieval. Foundations and Trends in Information Retrieval, 10(1), 1–117. https://doi.org/10.1561/1500000051
Hu, X., & Kando, N. (2017). Task complexity and difficulty in music information retrieval. Journal of the Association for Information Science and Technology, 68(7), 1711–1723. https://doi.org/10.1002/asi.23803
Jiang, J., He, D., Kelly, D., & Allan, J. (2017). Understanding ephemeral state of relevance. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (pp. 137–146). ACM. https://doi.org/10.1145/3020165.3020176
Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93(5), 1449–1475. https://doi.org/10.1257/000282803322655392
Kairam, S., Morris, M., Teevan, J., Liebling, D., & Dumais, S. (2013). Towards supporting search over trending events with social media. In Proceedings of the International AAAI Conference on Web and Social Media (Vol. 7, No. 1, pp. 283–292).
Kim, K. S., Sin, S. C. J., & He, Y. (2013). Information seeking through social media: Impact of user characteristics on social media use. Proceedings of the American Society for Information Science and Technology, 50(1), 1–4. https://doi.org/10.1002/meet.14505001155
Koskela, M., Luukkonen, P., Ruotsalo, T., Sjöberg, M., & Floréen, P. (2018). Proactive information retrieval by capturing search intent from primary task context. ACM Transactions on Interactive Intelligent Systems (TIIS), 8(3), 1–25. https://doi.org/10.1145/3150975
Liu, J. (2021). Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors. Information Processing & Management, 58(3), 102522. https://doi.org/10.1016/j.ipm.2021.102522
Liu, J. (2022). Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation. Information Processing & Management, 59(5), 103007. https://doi.org/10.1016/j.ipm.2022.103007
Liu, J., & Han, F. (2020). Investigating reference dependence effects on user search interaction and satisfaction: A behavioral economics perspective. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1141–1150). ACM. https://doi.org/10.1145/3397271.3401085
Liu, J., Liu, C., & Belkin, N. J. (2020b). Personalization in text information retrieval: A survey. Journal of the Association for Information Science and Technology, 71(3), 349–369. https://doi.org/10.1002/asi.24234
Liu, M., Mao, J., Liu, Y., Zhang, M., & Ma, S. (2019b). Investigating cognitive effects in session-level search user satisfaction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 923–931). ACM. https://doi.org/10.1145/3292500.3330981
Liu, J., Mitsui, M., Belkin, N. J., & Shah, C. (2019a). Task, information seeking intentions, and user behavior: Toward a multi-level understanding of Web search. In Proceedings of the 2019 ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 123–132). ACM. https://doi.org/10.1145/3295750.3298922
Liu, J., Sarkar, S., & Shah, C. (2020a). Identifying and predicting the states of complex search tasks. In Proceedings of the 2020 ACM SIGIR Conference on Human Information Interaction and Retrieval (pp. 193–202). ACM. https://doi.org/10.1145/3343413.3377976
Liu, J., & Shah, C. (2019a). Interactive IR user study design, evaluation, and reporting. Synthesis Lectures on Information Concepts, Retrieval, and Services, 11(2), i–93. https://doi.org/10.2200/S00923ED1V01Y201905ICR067
Liu, J., & Shah, C. (2019b). Proactive identification of query failure. Proceedings of the Association for Information Science and Technology, 56(1), 176–185. https://doi.org/10.1002/pra2.15
Liu, J., & Shah, C. (2022). Leveraging user interaction signals and task state information in adaptively optimizing usefulness-oriented search sessions. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (pp. 1–11). ACM. https://doi.org/10.1145/3529372.3530926
Liu, J., & Yu, R. (2021). State-aware meta-evaluation of evaluation metrics in interactive information retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (pp. 3258–3262). ACM. https://doi.org/10.1145/3459637.3482190
Luo, C., Liu, Y., Sakai, T., Zhou, K., Zhang, F., Li, X., & Ma, S. (2017). Does document relevance affect the searcher’s perception of time? In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (pp. 141–150). ACM. https://doi.org/10.1145/3018661.3018694
Luo, J., Zhang, S., & Yang, H. (2014). Win-win search: Dual-agent stochastic game in session search. In Proceedings of the 37th International ACM SIGIR conference on Research & Development in Information Retrieval (pp. 587–596). ACM. https://doi.org/10.1145/2600428.2609629
Mao, J., Liu, Y., Zhou, K., Nie, J. Y., Song, J., Zhang, M., Ma, S., Sun, J., & Luo, H. (2016). When does relevance mean usefulness and user satisfaction in web search? In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 463–472). ACM. https://doi.org/10.1145/2911451.2911507
Markle, A., Wu, G., White, R., & Sackett, A. (2018). Goals as reference points in marathon running: A novel test of reference dependence. Journal of Risk and Uncertainty, 56(1), 19–50. https://doi.org/10.1007/s11166-018-9271-9
Martin, V. (2017). When to quit: Narrow bracketing and reference dependence in taxi drivers. Journal of Economic Behavior & Organization, 144, 166–187. https://doi.org/10.1016/j.jebo.2017.09.024
Mitsui, M., Liu, J., Belkin, N. J., & Shah, C. (2017). Predicting information seeking intentions from search behaviors. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1121–1124). ACM. https://doi.org/10.1145/3077136.3080737
Mitsui, M., Liu, J., & Shah, C. (2018). How much is too much? Whole session vs. first query behaviors in task type prediction. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (pp. 1141–1144). ACM. https://doi.org/10.1145/3209978.3210105
Moffat, A., Bailey, P., Scholer, F., & Thomas, P. (2017). Incorporating user expectations and behavior into the measurement of search effectiveness. ACM Transactions on Information Systems (TOIS), 35(3), 1–38. https://doi.org/10.1145/3052768
Moffat, A., & Zobel, J. (2008). Rank-biased precision for measurement of retrieval effectiveness. ACM Transactions on Information Systems (TOIS), 27(1), 1–27. https://doi.org/10.1145/1416950.1416952
Nelson, T. E., Oxley, Z. M., & Clawson, R. A. (1997). Toward a psychology of framing effects. Political Behavior, 19(3), 221–246. https://doi.org/10.1023/A:1024834831093
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175
O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American society for Information Science and Technology, 59(6), 938–955. https://doi.org/10.1002/asi.20801
Oeldorf-Hirsch, A., Hecht, B., Morris, M. R., Teevan, J., & Gergle, D. (2014). To search or to ask: The routing of information needs between traditional search engines and social networks. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 16–27). ACM. https://doi.org/10.1145/2531602.2531706
Pirolli, P., & Card, S. (1999). Information foraging. Psychological Review, 106(4), 643–675. https://doi.org/10.1037/0033-295X.106.4.643
Pratt, J. W. (1978). Risk aversion in the small and in the large. In Uncertainty in economics (pp. 59–79). Academic Press. https://doi.org/10.1016/B978-0-12-214850-7.50010-3
Sanderson, M. (2010). Test collection based evaluation of information retrieval systems. Foundations and Trends in Information Retrieval, 4(4), 247–375. https://doi.org/10.1561/1500000009
Schmitt-Beck, R. (2015). Bandwagon effect. The International Encyclopedia of Political Communication, 1–5. https://doi.org/10.1002/9781118541555.wbiepc015
Scholer, F., Kelly, D., Wu, W. C., Lee, H. S., & Webber, W. (2013). The effect of threshold priming and need for cognition on relevance calibration and assessment. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 623–632). ACM. https://doi.org/10.1145/2484028.2484090
Sels, L., Ceulemans, E., & Kuppens, P. (2019). All’s well that ends well? A test of the peak-end rule in couples’ conflict discussions. European Journal of Social Psychology, 49(4), 794–806. https://doi.org/10.1002/ejsp.2547
Shah, C. (2018). Information fostering-being proactive with information seeking and retrieval: Perspective paper. In Proceedings of the 2018 International ACM SIGIR Conference on Human Information Interaction & Retrieval (pp. 62–71). ACM. https://doi.org/10.1145/3176349.3176389
Shah, C., & González-Ibáñez, R. (2011). Evaluating the synergic effect of collaboration in information seeking. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 913–922). ACM. https://doi.org/10.1145/2009916.2010038
Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118. https://doi.org/10.2307/1884852
Syed, R., & Collins-Thompson, K. (2018). Exploring document retrieval features associated with improved short- and long-term vocabulary learning outcomes. In Proceedings of the 2018 ACM SIGIR Conference on Human Information Interaction & Retrieval (pp. 191–200). ACM. https://doi.org/10.1145/3176349.3176397
Tiefenbeck, V., Goette, L., Degen, K., Tasic, V., Fleisch, E., Lalive, R., & Staake, T. (2018). Overcoming salience bias: How real-time feedback fosters resource conservation. Management Science, 64(3), 1458–1476. https://doi.org/10.1287/mnsc.2016.2646
Tipper, S. P. (1985). The negative priming effect: Inhibitory priming by ignored objects. The Quarterly Journal of Experimental Psychology, 37(4), 571–590. https://doi.org/10.1080/14640748508400920
Tversky, A., & Kahneman, D. (1991). Loss aversion in riskless choice: A reference-dependent model. The Quarterly Journal of Economics, 106(4), 1039–1061. https://doi.org/10.2307/2937956
Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926
Voorhees, E. M. (2008). On test collections for adaptive information retrieval. Information Processing & Management, 44(6), 1879–1885. https://doi.org/10.1016/j.ipm.2007.12.011
Vuong, T., Jacucci, G., & Ruotsalo, T. (2017). Proactive information retrieval via screen surveillance. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1313–1316). ACM. https://doi.org/10.1145/3077136.3084151
White, R. (2013). Beliefs and biases in web search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 3–12). ACM. https://doi.org/10.1145/2484028.2484053
White, R. W. (2016). Interactions with search systems. Cambridge University Press.
White, R. W., & Huang, J. (2010). Assessing the scenic route: Measuring the value of search trails in web logs. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 587–594). ACM. https://doi.org/10.1145/1835449.1835548
Wixom, B. H., & Todd, P. A. (2005). A theoretical integration of user satisfaction and technology acceptance. Information Systems Research, 16(1), 85–102. https://doi.org/10.1287/isre.1050.0042
Yu, R., Gadiraju, U., Holtz, P., Rokicki, M., Kemkes, P., & Dietze, S. (2018). Predicting user knowledge gain in informational search sessions. In Proceedings of the 41st ACM SIGIR Conference on Research & Development in Information Retrieval (pp. 75–84). ACM. https://doi.org/10.1145/3209978.3210064
Zehlike, M., Bonchi, F., Castillo, C., Hajian, S., Megahed, M., & Baeza-Yates, R. (2017). FA* IR: A fair Top-k ranking algorithm. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 1569–1578). ACM. https://doi.org/10.1145/3132847.3132938
Zhang, J., Liu, Y., Mao, J., Xie, X., Zhang, M., Ma, S., & Tian, Q. (2022). Global or local: Constructing personalized click models for Web search. In Proceedings of the ACM Web Conference (pp. 213–223). ACM. https://doi.org/10.1145/3485447.3511950
Zhang, Y., Liu, X., & Zhai, C. (2017). Information retrieval evaluation as search simulation: A general formal framework for IR evaluation. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (pp. 193–200). ACM. https://doi.org/10.1145/3121050.3121070
Zhang, F., Mao, J., Liu, Y., Ma, W., Zhang, M., & Ma, S. (2020b). Cascade or recency: Constructing better evaluation metrics for session search. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 389–398). ACM. https://doi.org/10.1145/3397271.3401163
Zhang, T., & Zhang, D. (2007). Agent-based simulation of consumer purchase decision-making and the decoy effect. Journal of Business Research, 60(8), 912–922. https://doi.org/10.1016/j.jbusres.2007.02.006
Zhang, W., Zhao, X., Zhao, L., Yin, D., Yang, G. H., & Beutel, A. (2020a). Deep reinforcement learning for information retrieval: Fundamentals and advances. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 2468–2471). ACM. https://doi.org/10.1145/3397271.3401467
Zviran, M., & Erlich, Z. (2003). Measuring IS user satisfaction: Review and implications. Communications of the Association for Information Systems, 12(1), 5. 10.17705/1CAIS.01205.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Liu, J. (2023). From Rational Agent to Human with Bounded Rationality. In: A Behavioral Economics Approach to Interactive Information Retrieval. The Information Retrieval Series, vol 48. Springer, Cham. https://doi.org/10.1007/978-3-031-23229-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-23229-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23228-2
Online ISBN: 978-3-031-23229-9
eBook Packages: Computer ScienceComputer Science (R0)