Closed-Loop Adaptive Decision Support Based on Automated Trust Assessment

  • Peter-Paul van Maanen
  • Tomas Klos
  • Kees van Dongen1
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4565)


This paper argues that it is important to study issues concerning trust and reliance when developing systems that are intended to augment cognition. Operators often under-rely on the help of a support system that provides advice or that performs certain cognitive tasks autonomously. The decision to rely on support seems to be largely determined by the notion of relative trust. However, this decision to rely on support is not always appropriate, especially when support systems are not perfectly reliable. Because the operator’s reliability estimations are typically imperfectly aligned or calibrated with the support system’s true capabilities, we propose that the aid makes an estimation of the extent of this calibration (under different circumstances) and intervenes accordingly. This system is intended to improve overall performance of the operator-support system as a whole. The possibilities in terms of application of these ideas are explored and an implementation of this concept in an abstract task environment has been used as a case study.


Adaptive System Task Allocation Reliance Decision Relative Trust Adaptive Support 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, J.D., See, K.A.: Trust in automation: Designing for appropriate reliance. Human Factors 46, 50–80 (2004)CrossRefGoogle Scholar
  2. 2.
    Lee, J.D., Moray, N.: Trust, control strategies and allocation of function in human machine systems. Ergonomics 22, 671–691 (1992)Google Scholar
  3. 3.
    Wiegmann, D.A., Rich, A., Zhang, H.: Automated diagnostic aids: the effects of aid reliability on user’s trust and reliance. Theoretical Issues in Ergonomics Science 2, 352–367 (2001)CrossRefGoogle Scholar
  4. 4.
    Klos, T.B., La Poutré, H.: A versatile approach to combining trust values for making binary decisions. In: Stølen, K., Winsborough, W.H., Martinelli, F., Massacci, F. (eds.) iTrust 2006. LNCS, vol. 3986, pp. 206–220. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    van Maanen, P.-P., van Dongen, K.: Towards task allocation decision support by means of cognitive modeling of trust. In: Proceedings of the Eighth International Workshop on Trust in Agent Societies (Trust 2005), pp. 168–77. Utrecht, The Netherlands (July 2005)Google Scholar
  6. 6.
    van Dongen, K., van Maanen, P.-P.: Under-reliance on the decision aid: A difference in calibration and attribution between self and aid. In: Proceedings of the Human Factors and Ergonomics Society’s 50th Annual Meeting, San Francisco, USA (2006)Google Scholar
  7. 7.
    Parasuraman, R., Mouloua, M., Hilburn, B.: Adaptive aiding and adaptive task allocation enhance human-machine interaction. Automation Technology and Human Performance: Current Research and Trends 22, 119–123 (1999)Google Scholar
  8. 8.
    Parasuraman, R., Riley, V.A.: Humans and automation: Use, misuse, disuse, abuse. Human Factors 39, 230–253 (1997)CrossRefGoogle Scholar
  9. 9.
    Moray, N., Inagaki, T., Itoh, M.: Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied 6, 44–58 (2000)CrossRefGoogle Scholar
  10. 10.
    Dzindolet, M.T., Peterson, S.A., Pomranky, R.A., Pierce, L.G., Beck, H.P.: The role of trust in automation reliance. International Journal of Human-Computer Studies 58, 697–718 (2003)CrossRefGoogle Scholar
  11. 11.
    van Dongen, K., van Maanen, P.-P.: Designing for dynamic task allocation. In: Proceedings of the Seventh International Naturalistic Decision Making Conference (NDM7), Amsterdam, The Netherlands (2005)Google Scholar
  12. 12.
    D’Agostini, G.: Bayesian inference in processing experimental data: Principles and basic applications. Reports on Progress in Physics 66, 1383–1419 (2003)CrossRefGoogle Scholar
  13. 13.
    Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B.: Bayesian Data Analysis. 2nd edn. Chapman & Hall/CRC (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Peter-Paul van Maanen
    • 1
    • 2
  • Tomas Klos
    • 3
  • Kees van Dongen1
    • 1
  1. 1.TNO Human Factors, P.O. Box 23, 3769 ZG SoesterbergThe Netherlands
  2. 2.Department of Artificial Intelligence, Vrije Universiteit Amsterdam, De Boelelaan 1081a, 1081 HV AmsterdamThe Netherlands
  3. 3.Dutch National Research Institute for Mathematics and Computer Science (CWI), P.O. Box 94079, 1090 GB AmsterdamThe Netherlands

Personalised recommendations