Abstract
AI-enabled services, such as chatbots and generative systems, are often unable to generate correct information per user request, thus creating user resistance and preventing the smooth diffusion of AI services. Previous research has mostly addressed how to improve AI responses but fails to consider user resistance against misinformation from AI. Based on inoculation theory and a heuristic systematic model, this chapter discusses the cognitive mechanisms of inoculation effects on using AI chatbots by addressing questions on how users construe inoculation messages and how the messages influence users’ resistance against misinformation. How inoculation confers resistance to users provides important implications for theory and practice. The chapter found that inoculation messages alleviate the negative effects of misinformation from AI chatbots on user interaction. It renders a critical perspective of how the theory can be conceptually extended to misinformation and how the theoretical frame can be used practically.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahluwalia, R., Unnava, H., & Burnkrant, R. (2001). The moderating role of commitment on the spillover effect of marketing communications. Journal of Marketing Research, 38(4), 458–470. https://doi.org/10.1509/jmkr.38.4.458.18903
Ahmed, S., & Gil-Lopez, T. (2022). Engaging with vilifying stereotypes. Journalism & Mass Communication Quarterly. https://doi.org/10.1177/10776990221110113
Ali, K., Li, C., Zain-ul-abdin, K., & Zaffar, M. (2022). Fake news on Facebook: Examining the impact of heuristic cues on perceived credibility. Internet Research, 32(1), 379–397. https://doi.org/10.1108/INTR-10-2019-0442
Barnoy, A., & Reich, Z. (2022). Trusting others: A Pareto distribution of source and message credibility among news reporters. Communication Research, 49(2), 196–220. https://doi.org/10.1177/0093650220911814
Basol, M., Roozenbeek, J., Berriche, M., Uenal, F., McClanahan, W., & Van der Linden, S. (2020). Toward psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data & Society, 8. https://doi.org/10.1177/20539517211013868
Basol, M., Roozenbeek, J., Berriche, M., Uenal, F., McClanahan, W., & Van der Linden, S. (2021). Toward psychological herd immunity: Cross-cultural evidence for two prebunking interventions against COVID-19 misinformation. Big Data & Society, 8. https://doi.org/10.1177/20539517211013868
Borukhson, D., Lorenz-Spreen, P., & Ragni, M. (2022). When does an individual accept misinformation? Computational Brain & Behavior, 5, 244–260. https://doi.org/10.1007/s42113-022-00136-3
Chaiken, S., Giner-Sorolla, R., & Chen, S. (1996). Beyond accuracy: Defense and impression motives in heuristic and systematic information processing. In P. M. Gollwitzer & J. A. Bargh (Eds.), The psychology of action: Linking cognition and motivation to behavior (pp. 553–578). Guilford Press.
Chen, Z. F., & Cheng, Y. (2020). Consumer response to fake news about brands on social media. Journal of Product & Brand Management, 29(2), 188–198. https://doi.org/10.1108/JPBM-12-2018-2145
Compton, J. (2021). Threat and/in Inoculation Theory. International. Journal of Communication, 15, 4294–4306. http://ijoc.org.JOSH.COMPTON
Compton, J. (2022). Threat and/in Inoculation Theory. International Journal of Communication 15, 4294–4306. Available at http://ijoc.org.JOSHCOMPTON
Compton, J., Linden, S., Cook, J., & Basol, M. (2021). Inoculation theory in the post-truth era: Extant findings and new frontiers for contested science, misinformation, and conspiracy theories. Social and Personality Psychology Compas, 15, e12602. https://doi.org/10.1111/spc3.12602
Compton, J., Linden, S., Cook, J., & Basol, M. (2022). Inoculation theory in the posttruth era: Extant findings and new frontiers for contested science, misinformation, and conspiracy theories. Social and Personality Psychology Compas, 15, e12602.
Compton, J., & Pfau, M. (2005). Inoculation theory of resistance to influence at maturity. Annals of International Communication Association, 29, 97–145. https://doi.org/10.1207/s15567419cy2901_4
Cronbach, L. (1989). Construct validation after thirty years. In R. L. Linn (Ed.), Intelligence: Measurement, theory and public policy (pp. 147–171). University of Illinois Press.
Diakopoulos, N., & Koliska, M. (2017). Algorithmic transparency in the news media. Digital Journalism, 5(7), 809–828. https://doi.org/10.1080/21670811.2016.1208053
Ecker, U., Lewandowsky, S., Cook, J. et al. (2021). The psychological drivers of misinformation belief and its resistance to correction. Nature Review Psychology, 1, 13–29. https://doi.org/10.1038/s44159-021-00006-y
Ecker, U., Lewandowsky, S., Cook, J., et al. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Review Psychology, 1, 13–29. https://doi.org/10.1038/s44159-021-00006-y
Epstein, Z., Berinsky, A., Cole, R., Gully, A., Pennycook, G., & Rand, D. (2021). Developing an accuracy-prompt toolkit to reduce COVID-19 misinformation online. Harvard Kennedy School Misinformation Review, 2(3), 1–12. https://doi.org/10.37016/mr-2020-71
Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education. JMIR Medical Education, 9, e46885. https://doi.org/10.2196/46885
Feezell, J., Wagner, J., & Conroy, M. (2021). Exploring the effects of algorithm-driven news sources on political behavior and polarization. Computers in Human Behavior, 116, 106626. https://doi.org/10.1016/j.chb.2020.106626
Gran, A., Booth, P., & Bucher, T. (2021). To be or not to be algorithm aware. Information, Communication & Society, 24(12), 1779–1796. https://doi.org/10.1080/1369118X.2020.1736124
Hair, J. F., Jr., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis (3rd ed.). Macmillan Publishing Company.
Jia, C., & Johnson, T. (2021). Source credibility matters: Does automated journalism inspire selective exposure? International Journal of Communication, 15(1), 1–19.
Joreskog, K., & Sorbom, D. (1996). LISREL 8: User’s reference guide. Scientific Software International.
Korzynski, P., Mazurek, G., Altmann, A., Ejdys, J., Kazlauskaite, R., Paliszkiewicz, J., Wach, K., & Ziemba, E. (2023). Generative artificial intelligence as a new context for management theories: Analysis of ChatGPT. Central European Management Journal. https://doi.org/10.1108/CEMJ-02-2023-0091
Margolin, D. (2021). The theory of informative fictions. Communication Theory, 31(4), 714–736. https://doi.org/10.1093/ct/qtaa002
Martel, C., Pennycook, G., & Rand, D. G. (2020). Reliance on emotion promotes belief in fake news. Cognitive Research, 47(5). https://doi.org/10.1186/s41235-020-00252-3
Melchior, C., & Oliveira, M. (2022). Health-related fake news on social media platforms. New Media & Society, 24(6), 1500–1522. https://doi.org/10.1177/14614448211038762
Mhasawade, V., Zhao, Y., & Chunara, R. (2021). Machine learning and algorithmic fairness in public and population health. Nature Machine Intelligence, 3, 659–666. https://doi.org/10.1038/s42256-021-00373-4
Molina, M., & Sundar, S. (2023). Does distrust in humans predict greater trust in AI? New Media & Society. https://doi.org/10.1177/14614448221103534
Musi, E., Carmi, E., Reed, C., Yates, S., & O’Halloran, K. (2023). Developing misinformation immunity: How to reason-check fallacious news in a human–computer interaction environment. Social Media + Society. https://doi.org/10.1177/20563051221150407
Niu, W., Huang, L., & Chen, M. (2021). Spanning from diagnosticity to serendipity. International Journal of Information Management, 60, 102362. https://doi.org/10.1016/j.ijinfomgt.2021.102362
Peifer, J., & Meisinger, J. (2021). The value of explaining the process. Journalism & Mass Communication Quarterly, 98(3), 828–853. https://doi.org/10.1177/10776990211012953
Pennycook, G. (2023). A framework for understanding reasoning errors. Advances in Experimental Social Psychology, 67, 131–208. https://doi.org/10.1016/bs.aesp.2022.11.003
Pennycook, G., & Rand, D. G. (2022). Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation. Nature Communications, 13, 2333. https://doi.org/10.1038/s41467-022-30073-5
Roozenbeek, J., & Van der Linden, S. (2019). Technique-based inoculation against real-world misinformation. Research Society Open Science, 9(8), 211–719. https://doi.org/10.1098/rsos.211719
Roozenbeek, J., et al. (2022). Psychological inoculation improves resilience against misinformation on social media. Science. Advances, 8, eabo6254. https://doi.org/10.1126/sciadv.abo6254
Schuetz, S., Sykes, T., & Venkatesh, V. (2021). Combating COVID-19 fake news on social media through fact-checking. European Journal of Information Systems, 30(4), 376–388. https://doi.org/10.1080/0960085X.2021.1895682
Shin, D. (2020). User perceptions of algorithmic decisions in the personalized AI system: Perceptual evaluation of fairness, accountability, transparency, and explainability. Journal of Broadcasting & Electronic Media, 64(4), 541–565. https://doi.org/10.1080/08838151.2020.1843357
Shin, D. (2022). Expanding the role of trust in the experience of algorithmic journalism: User sensemaking of algorithmic heuristics in Korean users. Journalism Practice, 16(6), 1168–1191. https://doi.org/10.1080/17512786.2020.1841018
Shin, D. (2023). Algorithms, humans, and interactions. Routledge. https://doi.org/10.1201/b23083
Shin, D., & Akhtar, F. (2024). Algorithmic inoculation against misinformation: How to build cognitive immunity against misinformation. Journal of Broadcasting & Electronic Media. https://doi.org/10.1080/08838151.2024.2323712
Shin, D., Koerber, A., & Lim, J. (2024). Impact of misinformation from generative AI on user information processing: How people understand misinformation on generative AI. New Media and Society, 26(4), 12–29.
Stecula, D., Kuru, O., & Jamieson, K. (2020). How trust in experts and media use affect acceptance of common anti-vaccination claims. The Harvard Kennedy School Misinformation Review, 1(1), 1–11. https://doi.org/10.37016/mr-2020-007
Sundar, S., Knobloch-Westerwick, S., & Hastall, M. (2007). News cues: Information scent and cognitive heuristics. Journal of the American Society for Information Science and Technology, 58(3), 366–378. https://doi.org/10.1002/asi.20511
Traberg, C., Roozenbeek, J., & Van der Linden, S. (2022). Psychological inoculation against misinformation. The ANNALS of the American Academy of Political and Social Science, 700(1), 136–151. https://doi.org/10.1177/00027162221087936
Tully, M., Bode, L., & Vraga, E. (2020). Mobilizing users: Does exposure to misinformation and its correction affect users’ responses to a health misinformation post? Social Media + Society, 6(4). https://doi.org/10.1177/2056305120978377
Vafeiadis, M., & Xiao, A. (2021). Fake news: How emotions, involvement, need for cognition and rebuttal evidence influence consumer reactions toward a targeted organization. Public Relations Review, 47(4), 102088. https://doi.org/10.1016/j.pubrev.2021.102088
Van der Linden, S. (2023). Foolproof: Why misinformation infects our minds and how to build immunity. W.W. Norton & Company.
Van Dis, E., Bollen, J., Zuidema, W., Rooij, R., & Bockting, C. (2023). ChatGPT: Five priorities for research. Nature, 614, 224–226. https://doi.org/10.1038/d41586-023-00288-7
Vraga, E., & Bode, L. (2020). Defining misinformation and understanding its bounded nature. Political Communication, 37(1), 136–144. https://doi.org/10.1080/10584609.2020.1716500
Walter, N., & Tukachinsky, R. (2020). A meta-analytic examination of the continued influence of misinformation in the face of correction. Communication Research, 47, 155–177. https://doi.org/10.1177/0093650219854600
Wathen, C., & Burkell, J. (2002). Believe it or not: Factors influencing credibility on the web. Journal of the American Society for Information Science and Technology, 53(2), 134–144.
Zrnec, A., Pozenel, M., & Lavbic, D. (2022). Users’ ability to perceive misinformation. Information Processing & Management, 59(1), 102739. https://doi.org/10.1016/j.ipm.2021.102739
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Shin, D. (2024). Misinformation and Inoculation: Algorithmic Inoculation Against Misinformation Resistance. In: Artificial Misinformation. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-52569-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-52569-8_8
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-031-52568-1
Online ISBN: 978-3-031-52569-8
eBook Packages: Literature, Cultural and Media StudiesLiterature, Cultural and Media Studies (R0)