Skip to main content

In AI we trust? Perceptions about automated decision-making by artificial intelligence

Abstract

Fueled by ever-growing amounts of (digital) data and advances in artificial intelligence, decision-making in contemporary societies is increasingly delegated to automated processes. Drawing from social science theories and from the emerging body of research about algorithmic appreciation and algorithmic perceptions, the current study explores the extent to which personal characteristics can be linked to perceptions of automated decision-making by AI, and the boundary conditions of these perceptions, namely the extent to which such perceptions differ across media, (public) health, and judicial contexts. Data from a scenario-based survey experiment with a national sample (N = 958) show that people are by and large concerned about risks and have mixed opinions about fairness and usefulness of automated decision-making at a societal level, with general attitudes influenced by individual characteristics. Interestingly, decisions taken automatically by AI were often evaluated on par or even better than human experts for specific decisions. Theoretical and societal implications about these findings are discussed.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    While perceptions of ADM are relevant in many societal contexts, this study has chosen to focus on media, (public) health, and justice. In these three sectors, we expect that ADM can have a significant impact on individual rights, well-being, and functioning in a society (as citizens and voters in the case of the media, as members of a society in the case of justice, and as humans in the case of health).

  2. 2.

    Results are reported for the measure with higher reliability (without the reversed item), but differences are communicated in the notes (Table 2).

  3. 3.

    Approximately 28% (818) of the responses for all the scenarios combined (N = 2874) were removed because of the manipulation check. When running the analyses with these responses included, the results stay largely the same with regards to direction and significance levels. Exceptions are indicated in the notes.

References

  1. Agarwal R, Gao G, DesRoches C, Jha AK (2010) Research commentary—the digital transformation of healthcare: current status and the road ahead. Inf Syst Res 21:796–809. https://doi.org/10.1287/isre.1100.0327

    Article  Google Scholar 

  2. Angwin J, Larson J, Mattu S, Kirchner L (2016) Machine Bias. In: ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Accessed 18 Jun 2018

  3. Baek TH, Morimoto M (2012) Stay away from me. J Advert 41:59–76. https://doi.org/10.2753/JOA0091-3367410105

    Article  Google Scholar 

  4. Bickmore T, Utami D, Matsuyama R, Paasche-Orlow MK (2016) Improving access to online health information with conversational agents: a randomized controlled experiment. J Med Internet Res. https://doi.org/10.2196/jmir.5239

    Article  Google Scholar 

  5. Boerman SC, Kruikemeier S, Borgesius FJZ (2017) Online behavioral advertising: a literature review and research agenda. J Advert. https://doi.org/10.1080/00913367.2017.1339368

    Article  Google Scholar 

  6. Boerman SC, Kruikemeier S, Zuiderveen Borgesius FJ (2018) Exploring motivations for online privacy protection behavior: insights from panel data. Commun Res. https://doi.org/10.1177/0093650218800915

    Article  Google Scholar 

  7. Bol N, Kruikemeier S, Boerman SC et al (2018) Understanding the effects of personalization as a privacy calculus: analyzing self-disclosure across health, news, and commerce contexts. J Comput-Mediat Commun 23(6):370–388

    Article  Google Scholar 

  8. Carlson M (2018) Automating judgment? Algorithmic judgment, news knowledge, and journalistic professionalism. New Media Soc 20:1755–1772. https://doi.org/10.1177/1461444817706684

    Article  Google Scholar 

  9. Chu Z, Gianvecchio S, Wang H, Jajodia S (2012) Detecting automation of twitter accounts: Are you a human, bot, or cyborg? IEEE Trans Dependable Secure Comput 9:811–824. https://doi.org/10.1109/TDSC.2012.75

    Article  Google Scholar 

  10. Cox D, Cox AD (2001) Communicating the consequences of early detection: the role of evidence and framing. J Mark 65:91–103. https://doi.org/10.1509/jmkg.65.3.91.18336

    Article  Google Scholar 

  11. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13:319–340. https://doi.org/10.2307/249008

    Article  Google Scholar 

  12. Dawes RM, Faust D, Meehl PE (1989) Clinical versus actuarial judgment. Science 243:1668–1674. https://doi.org/10.1126/science.2648573

    Article  Google Scholar 

  13. Diakopoulos N, Koliska M (2017) Algorithmic transparency in the news media. Digit J 5:809–828. https://doi.org/10.1080/21670811.2016.1208053

    Article  Google Scholar 

  14. Dietvorst BJ, Simmons JP, Massey C (2015) Algorithm aversion: people erroneously avoid algorithms after seeing them err. J Exp Psychol Gen 144:114–126. https://doi.org/10.1037/xge0000033

    Article  Google Scholar 

  15. Dijkstra JJ, Liebrand WBG, Timminga E (1998) Persuasiveness of expert systems. Behav Inf Technol 17:155–163. https://doi.org/10.1080/014492998119526

    Article  Google Scholar 

  16. Dilsizian SE, Siegel EL (2013) Artificial intelligence in medicine and cardiac imaging: harnessing big data and advanced computing to provide personalized medical diagnosis and treatment. Curr Cardiol Rep 16:441. https://doi.org/10.1007/s11886-013-0441-8

    Article  Google Scholar 

  17. Dineen BR, Noe RA, Wang C (2004) Perceived fairness of web-based applicant screening procedures: weighing the rules of justice and the role of individual differences. Hum Resour Manage 43:127–145. https://doi.org/10.1002/hrm.20011

    Article  Google Scholar 

  18. Dodge M, Kitchin R (2007) The automatic management of drivers and driving spaces. Geoforum 38:264–275. https://doi.org/10.1016/j.geoforum.2006.08.004

    Article  Google Scholar 

  19. Dressel J, Farid H (2018) The accuracy, fairness, and limits of predicting recidivism. Sci Adv. https://doi.org/10.1126/sciadv.aao5580

    Article  Google Scholar 

  20. Elish MC, Boyd danah (2018) Situating methods in the magic of Big Data and AI. Commun Monogr 85:57–80. https://doi.org/10.1080/03637751.2017.1375130

    Article  Google Scholar 

  21. European Commission (2018) Data protection working party. Directive 95/46/EC § Articles 29 and 30

  22. Ferrara E, Varol O, Davis C et al (2016) The rise of social bots. Commun ACM 59:96–104. https://doi.org/10.1145/2818717

    Article  Google Scholar 

  23. Field A (2013) Discovering statistics using IBM SPSS statistics. Sage, Newcastle

    Google Scholar 

  24. Gillespie T (2014) The relevance of algorithms. Media Technol Essays Commun Mater Soc 167:167

    Google Scholar 

  25. Graefe A, Haim M, Haarmann B, Brosius H-B (2018) Readers’ perception of computer-generated news: credibility, expertise, and readability. Journalism 19:595–610. https://doi.org/10.1177/1464884916641269

    Article  Google Scholar 

  26. Grolleman J, van Dijk B, Nijholt A, van Emst A (2006) Break the Habit! designing an e-therapy intervention using a virtual coach in aid of smoking cessation. In: IJsselsteijn WA, de Kort YAW, Midden C et al (eds) Persuasive technology. Springer, Berlin, pp 133–141

    Chapter  Google Scholar 

  27. Hajian S, Bonchi F, Castillo C (2016) Algorithmic bias: from discrimination discovery to fairness-aware data mining. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD’16. ACM Press, San Francisco, California, USA, pp 2125–2126. https://doi.org/10.1145/2939672.2945386

  28. Hudlicka E (2013) Virtual training and coaching of health behavior: example from mindfulness meditation training. Patient Educ Couns 92:160–166. https://doi.org/10.1016/j.pec.2013.05.007

    Article  Google Scholar 

  29. Inglehart R, Haerpfer C, Moreno A, Welzel C, Kizilova K, Diez-Medrano J, Lagos M, Norris P, Ponarin E, Puranen B et al (eds) (2014). World values survey: round five-country-pooled datafile version. JD Systems Institute, Madrid. www.worldvaluessurvey.org/WVSDocumentationWV5.jsp

  30. Jha S, Topol EJ (2016) Adapting to artificial intelligence: radiologists and pathologists as information specialists. JAMA 316:2353–2354. https://doi.org/10.1001/jama.2016.17438

    Article  Google Scholar 

  31. Kennedy LW, Caplan JM, Piza E (2011) Risk clusters, hotspots, and spatial intelligence: risk terrain modeling as an algorithm for police resource allocation strategies. J Quant Criminol 27:339–362. https://doi.org/10.1007/s10940-010-9126-2

    Article  Google Scholar 

  32. Kitchin R (2017) Thinking critically about and researching algorithms. Inf Commun Soc 20:14–29. https://doi.org/10.1080/1369118X.2016.1154087

    Article  Google Scholar 

  33. Lee MK (2018) Understanding perception of algorithmic decisions: fairness, trust, and emotion in response to algorithmic management. Big Data Soc 5:2053951718756684. https://doi.org/10.1177/2053951718756684

    Article  Google Scholar 

  34. Lee MK, Baykal S (2017) Algorithmic Mediation in Group Decisions: Fairness Perceptions of Algorithmically Mediated vs. Discussion-Based Social Division. ACM Press, pp 1035–1048

  35. Logg J (2017) Theory of machine: When do people rely on algorithms? SSRN Electron J. https://doi.org/10.2139/ssrn.2941774

    Article  Google Scholar 

  36. Logg J, Minson J, Moore DA (2018) Algorithm appreciation: people prefer algorithmic to human judgment. Social Science Research Network, Rochester

    Google Scholar 

  37. Madhavan P, Wiegmann DA (2007) Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Hum Factors J Hum Factors Ergon Soc 49:773–785. https://doi.org/10.1518/001872007X230154

    Article  Google Scholar 

  38. McQuillan D (2015) Algorithmic states of exception. Eur J Cult Stud 18:564–576. https://doi.org/10.1177/1367549415577389

    Article  Google Scholar 

  39. Nass C, Steuer J, Tauber ER (1994) Computers are social actors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp 72–78

  40. Newell S, Marabelli M (2015) Strategic opportunities (and challenges) of algorithmic decision-making: a call for action on the long-term societal effects of ‘datification’. J Strateg Inf Syst 24:3–14. https://doi.org/10.1016/j.jsis.2015.02.001

    Article  Google Scholar 

  41. Nissan E (2017) Digital technologies and artificial intelligence’s present and foreseeable impact on lawyering, judging, policing and law enforcement. AI Soc 32:441–464. https://doi.org/10.1007/s00146-015-0596-5

    Article  Google Scholar 

  42. Nysveen H (2005) Intentions to use mobile services: antecedents and cross-service comparisons. J Acad Mark Sci 33:330–346. https://doi.org/10.1177/0092070305276149

    Article  Google Scholar 

  43. O’Neil C (2017) Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books

  44. Pierson E (2017) Demographics and discussion influence views on algorithmic fairness. ArXiv:1712.09124 Cs

  45. Siddiqui H, Healy E, Olmsted A (2017) Bot or not. In: 2017 12th international conference for internet technology and secured transactions (ICITST). pp 462–463. https://doi.org/10.23919/ICITST.2017.8356448

  46. Smith A (2018) Public attitudes toward computer algorithms. pp 1–40. Retrieved from Pew Research Center website http://www.pewinternet.org/2018/11/16/public-attitudes-toward-computer-algorithms/

  47. Stanford University (2016) One hundred year study on artificial intelligence (AI). Retrieved from https://ai100.stanford.edu/

  48. Sundar SS (2008) The MAIN model: a heuristic approach to understanding technology effects on credibility. In: Metzger MJ, Flanagin AJ (eds) Digital media, youth, and credibility. MIT Press, Cambridge, MA, pp 73–100

    Google Scholar 

  49. Sundar SS, Nass C (2000) Source orientation in human-computer interaction programmer, networker, or independent social actor. Commun Res 27:683–703. https://doi.org/10.1177/009365000027006001

    Article  Google Scholar 

  50. Sundar SS, Nass C (2001) Conceptualizing sources in online news. J Commun 51:52–72

    Article  Google Scholar 

  51. Thurman N, Schifferes S (2012) The future of personalization at news websites. J Stud 13:775–790. https://doi.org/10.1080/1461670X.2012.664341

    Article  Google Scholar 

  52. Thurman N, Moeller J, Helberger N, Trilling D (2018) My friends, editors, algorithms, and I. Digit J. https://doi.org/10.1080/21670811.2018.1493936

    Article  Google Scholar 

  53. van Dijck J, Poell T, de Waal M (2018) The platform society: public values in a connective world. Oxford University Press, Oxford

    Book  Google Scholar 

  54. Yeomans M, Shah A, Mullainathan S, Kleinberg J (2019) Making sense of recommendations. J Behav Decis Making 32(4):403–414. https://doi.org/10.1002/bdm.2118

    Article  Google Scholar 

  55. Yu K-H, Kohane IS (2018) Framing the challenges of artificial intelligence in medicine. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2018-008551

    Article  Google Scholar 

  56. Zarsky T (2016) The trouble with algorithmic decisions: an analytic road map to examine efficiency and fairness in automated and opaque decision making. Sci Technol Hum Values 41:118–132. https://doi.org/10.1177/0162243915605575

    Article  Google Scholar 

Download references

Acknowledgements

This study was funded by the Research Priority Area Communication and its Digital Communication Methods Lab at the University of Amsterdam.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Theo Araujo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Araujo, T., Helberger, N., Kruikemeier, S. et al. In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI & Soc 35, 611–623 (2020). https://doi.org/10.1007/s00146-019-00931-w

Download citation

Keywords

  • Automated decision-making
  • Artificial intelligence
  • Algorithmic fairness
  • Algorithmic appreciation
  • User perceptions