Effects of Uncertainty and Cognitive Load on User Trust in Predictive Decision Making

  • Jianlong Zhou
  • Syed Z. Arshad
  • Simon Luo
  • Fang Chen
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10516)

Abstract

Rapid increase of data in different fields has been resulting in wide applications of Machine Learning (ML) based intelligent systems in predictive decision making scenarios. Unfortunately, these systems appear like a ‘black-box’ to users due to their complex working mechanisms and therefore significantly affect the user’s trust in human-machine interactions. This is partly due to the tightly coupled uncertainty inherent in the ML models that underlie the predictive decision making recommendations. Furthermore, when such analytics-driven intelligent systems are used in modern complex high-risk domains (such as aviation) - user decisions, in addition to trust, are also influenced by higher levels of cognitive load. This paper investigates effects of uncertainty and cognitive load on user trust in predictive decision making in order to design effective user interfaces for such ML-based intelligent systems. Our user study of 42 subjects in a repeated factorial design experiment found that both uncertainty types (risk and ambiguity) and cognitive workload levels affected user trust in predictive decision making. Uncertainty presentation leads to increased trust but only under low cognitive load conditions when users had sufficient cognitive resources to process the information. Presentation of uncertainty under high load conditions (when cognitive resources were short in supply) leads to a decrease of trust in the system and its recommendations.

Keywords

Trust Uncertainty Cognitive load Predictive decision making 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

Authors thank all volunteer participants for the experiment. This work was supported in part by the Asian Office of Aerospace Research & Development (AOARD) under grant No. FA2386-14-1-0022 AOARD 134131.

References

  1. 1.
    Hoff, K.A., Bashir, M.: Trust in automation integrating empirical evidence on factors that influence trust. Hum. Factors J. Hum. Factors Ergon. Soc. 57, 407–434 (2015). doi: 10.1177/0018720814547570 CrossRefGoogle Scholar
  2. 2.
    Marusich, L.R., Bakdash, J.Z., Onal, E., et al.: Effects of information availability on command-and-control decision making performance, trust, and situation awareness. Hum. Factors J. Hum. Factors Ergon. Soc. 58, 301–321 (2016). doi: 10.1177/0018720815619515 CrossRefGoogle Scholar
  3. 3.
    Mayer, R.C., Davis, J.H., Schoorman, F.D.: An integrative model of organizational trust. Acad. Manage. Rev. 20, 709–734 (1995). doi: 10.5465/AMR.1995.9508080335 Google Scholar
  4. 4.
    Schoorman, F.D., Mayer, R.C., Davis, J.H.: An integrative model of organizational trust: past, present, and future. Acad. Manage. Rev. 32, 344–354 (2007). doi: 10.5465/AMR.2007.24348410 CrossRefGoogle Scholar
  5. 5.
    Wheeler, S.: Trusted Autonomy: Concept Development in Technology Foresight. Defense Technical Information Center, Defense Science & Technology Group, Australia (2015)Google Scholar
  6. 6.
    Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors 46, 50–80 (2004)CrossRefGoogle Scholar
  7. 7.
    Mercado, J.E., Rupp, M.A., Chen, J.Y.C., et al.: Intelligent agent transparency in human-agent teaming for Multi-UxV management. Hum. Factors 58, 401–415 (2016)CrossRefGoogle Scholar
  8. 8.
    Ososky, S., Sanders, T., Jentsch, F., et al.: Determinants of system transparency and its influence on trust in and reliance on unmanned robotic systems. In: Karlsen, R.E., Gage, D.W., Shoemaker, C.M., Gerhart, G.R. (eds.), p. 90840E (2014)Google Scholar
  9. 9.
    Adobor, H.: Optimal trust? Uncertainty as a determinant and limit to trust in inter-firm alliances. Leadersh. Org Dev. J. 27, 537–553 (2006). doi: 10.1108/01437730610692407 CrossRefGoogle Scholar
  10. 10.
    Parasuraman, R., Sheridan, T.B., Wickens, D.C.: Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs. J. Cogn. Eng. Decis. Making 2, 140–160 (2008)CrossRefGoogle Scholar
  11. 11.
    Khawaji, A., Chen, F., Zhou, J., Marcus, N.: Trust and cognitive load in the text-chat environment: the role of mouse movement. In: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, pp. 324–327 (2014)Google Scholar
  12. 12.
    Smith, P.J., Geddes, N.D., Beatty, R.: Human-centered design of decision-support systems. In: Human-Computer Interaction: Design Issues, Solutions, and Applications (2009)Google Scholar
  13. 13.
    Zhou, J., Li, Z., Wang, Y., Chen, F.: Transparent machine learning — revealing internal states of machine learning. In: Proceedings of IUI 2013 Workshop on Interactive Machine Learning (2013)Google Scholar
  14. 14.
    Zhou, J., Khawaja, M.A., Li, Z., et al.: Making machine learning useable by revealing internal states update — a transparent approach. Int. J. Comput. Sci. Eng. 13, 378–389 (2016)CrossRefGoogle Scholar
  15. 15.
    Winkler, R.L.: The importance of communicating uncertainties in forecasts: overestimating the risks from winter storm Juno. Risk Anal. 35, 349–353 (2015)CrossRefGoogle Scholar
  16. 16.
    Beller, J., Heesen, M., Vollrath, M.: Improving the driver-automation interaction an approach using automation uncertainty. Hum. Factors J. Hum. Factors Ergon. Soc. 55, 1130–1141 (2013). doi: 10.1177/0018720813482327 CrossRefGoogle Scholar
  17. 17.
    Candello, H.: User methods and approaches to design cognitive systems. In: Marcus, A. (ed.) DUXU 2016, Part I. LNCS, vol. 9746, pp. 231–242. Springer, Cham (2016). doi: 10.1007/978-3-319-40409-7_23 CrossRefGoogle Scholar
  18. 18.
    Arshad, S.Z., Zhou, J., Bridon, C., et al.: Investigating user confidence for uncertainty presentation in predictive decision making. In: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, pp. 352–360. ACM, New York (2015)Google Scholar
  19. 19.
    Rouse, W.B.: Adaptive aiding for human/computer control. Hum. Factors J. Hum. Factors Ergon. Soc. 30, 431–443 (1988). doi: 10.1177/001872088803000405 CrossRefGoogle Scholar
  20. 20.
    Byrne, E.A., Parasuraman, R.: Psychophysiology and adaptive automation. Biol. Psychol. 42, 249–268 (1996)CrossRefGoogle Scholar
  21. 21.
    Moray, N., Inagaki, T., Itoh, M.: Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. J. Exp. Psychol. Appl. 6, 44–58 (2000)CrossRefGoogle Scholar
  22. 22.
    LeClerc, J., Joslyn, S.: The cry wolf effect and weather-related decision making. Risk Anal. 35, 385–395 (2015). doi: 10.1111/risa.12336 CrossRefGoogle Scholar
  23. 23.
    Uggirala, A., Gramopadhye, A.K., Melloy, B.J., Toler, J.E.: Measurement of trust in complex and dynamic systems using a quantitative approach. Int. J. Ind. Ergon. 34, 175–186 (2004)CrossRefGoogle Scholar
  24. 24.
    Allen, P.M., Edwards, J.A., Snyder, F.J., et al.: The effect of cognitive load on decision making with graphically displayed uncertainty information. Risk Anal. 34, 1495–1505 (2014)CrossRefGoogle Scholar
  25. 25.
    Ibrekk, H., Morgan, M.G.: Graphical communication of uncertain quantities to nontechnical people. Risk Anal. 7, 519–529 (1987)CrossRefGoogle Scholar
  26. 26.
    Edwards, J.A., Snyder, F.J., Allen, P.M., et al.: Decision making for risk management: a comparison of graphical methods for presenting quantitative uncertainty. Risk Anal. 32, 2055–2070 (2012). doi: 10.1111/j.1539-6924.2012.01839.x CrossRefGoogle Scholar
  27. 27.
    Kirschenbaum, S.S., Trafton, J.G., Schunn, C.D., Trickett, S.B.: Visualizing Uncertainty The Impact on Performance. Hum. Factors J. Hum. Factors Ergon. Soc. 56, 509–520 (2014). doi: 10.1177/0018720813498093 CrossRefGoogle Scholar
  28. 28.
    Damghani, K.K., Taghavifard, M.T., Moghaddam, R.T.: Decision Making Under Uncertain and Risky Situations. Enterprise Risk Management Symposium Monograph Society of Actuaries-Schaumburg, Illinois, vol. 15 (2009)Google Scholar
  29. 29.
    de Visser, E., Parasuraman, R.: Adaptive aiding of human-robot teaming effects of imperfect automation on performance, trust, and workload. J. Cogn. Eng. Decis. Making 5, 209–231 (2011). doi: 10.1177/1555343411410160 CrossRefGoogle Scholar
  30. 30.
    Li, Z., Zhang, B., Wang, Y., et al.: Water pipe condition assessment: a hierarchical beta process approach for sparse incident data. Mach. Learn. 95, 11–26 (2014)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Deck, C., Jahedi, S.: The effect of cognitive load on economic decision making: a survey and new experiments. Eur. Econ. Rev. 78, 97–119 (2015)CrossRefGoogle Scholar
  32. 32.
    Arshad, S.Z., Wang, Y., Chen, F.: Interactive mouse stream as real-time indicator of user’s cognitive load. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1025–1030. ACM, New York (2015)Google Scholar
  33. 33.
    Chen, F., Zhou, J., Wang, Y., et al.: Robust Multimodal Cognitive Load Measurement. Springer International Publishing, Cham (2016)CrossRefGoogle Scholar
  34. 34.
    Zhou, J., Sun, J., Chen, F., et al.: Measurable decision making with GSR and pupillary analysis for intelligent user interface. ACM Trans. Comput. Hum. Interact. 21, 33 (2015)CrossRefGoogle Scholar
  35. 35.
    Helgee, E.A.: Improving Drug Discovery Decision Making using Machine Learning and Graph Theory in QSAR Modeling. Ph.D. thesis, University of Gothenburg (2010)Google Scholar
  36. 36.
    Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014)CrossRefGoogle Scholar
  37. 37.
    Zhou, J., Chen, F.: Making machine learning useable. Int. J. Intell. Syst. Technol. Appl. 14, 91 (2015). doi: 10.1504/IJISTA.2015.074069 Google Scholar
  38. 38.
    Krause, J., Perer, A., Ng, K.: Interacting with predictions: visual inspection of black-box machine learning models. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, USA, pp. 5686–5697 (2016)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Jianlong Zhou
    • 1
  • Syed Z. Arshad
    • 1
  • Simon Luo
    • 1
  • Fang Chen
    • 1
  1. 1.DATA61, CSIROEveleighAustralia

Personalised recommendations