Incorporating Transparency During Trust-Guided Behavior Adaptation

  • Michael W. FloydEmail author
  • David W. Aha
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9969)


An important consideration in human-robot teams is ensuring that the robot is trusted by its teammates. Without adequate trust, the robot may be underutilized or disused, potentially exposing human teammates to dangerous situations. We have previously investigated an agent that can assess its own trustworthiness and adapt its behavior accordingly. In this paper we extend our work by adding a transparency layer that allows the agent to explain why it adapted its behavior. The agent uses explanations based on explicit feedback received from an operator. This allows it to provide simple, concise, and understandable explanations. We evaluate our system on scenarios from a simulated robotics domain by demonstrating that the agent can provide explanations that closely align with an operator’s feedback.


Inverse trust Behavior adaptation Explanation Transparency 



Thanks to ONR for sponsoring this research. Thanks also to Michael Drinkwater for his assistance in developing the eBotworks scenarios we used to evaluate our agent, and to the reviewers for their comments.


  1. 1.
    Floyd, M.W., Drinkwater, M., Aha, D.W.: How much do you trust me? Learning a case-based model of inverse trust. In: Lamontagne, L., Plaza, E. (eds.) ICCBR 2014. LNCS, vol. 8765, pp. 125-139. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11209-1_10 Google Scholar
  2. 2.
    Floyd, M.W., Drinkwater, M., Aha, D.W.: Improving trust-guided behavior adaptation using operator feedback. In: Hüllermeier, E., Minor, M. (eds.) ICCBR 2015. LNCS, vol. 9343, pp. 134-148. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24586-7_10 CrossRefGoogle Scholar
  3. 3.
    Dzindolet, M.T., Peterson, S.A., Pomranky, R.A., Pierce, L.G., Beck, H.P.: The role of trust in automation reliance. Int. J. Hum Comput Stud. 58(6), 697-718 (2003)CrossRefGoogle Scholar
  4. 4.
    Sabater, J., Sierra, C.: Review on computational trust and reputation models. Artif. Intell. Rev. 24(1), 33-60 (2005)CrossRefzbMATHGoogle Scholar
  5. 5.
    Hancock, P.A., Billings, D.R., Schaefer, K.E., Chen, J.Y., De Visser, E.J., Parasuraman, R.: A meta-analysis of factors affecting trust in human-robot interaction. Hum. Factors J. Hum. Factors Ergon. Soc. 53(5), 517-527 (2011)CrossRefGoogle Scholar
  6. 6.
    Kaniarasu, P., Steinfeld, A., Desai, M., Yanco, H.A.: Potential measures for detecting trust changes. In: Proceedings of the Seventh International Conference on Human-Robot Interaction, pp. 241-242. ACM, Boston (2012)Google Scholar
  7. 7.
    Knexus Research Corporation: eBotworks (2016). Retrieved from
  8. 8.
    Chen, J.Y.C., Barnes, M.J., Selkowitz, A.R., Stowers, K., Lakhmani, S.G., Kasdaglis, N.: Human-autonomy teaming and agent transparency. In: Proceedings of the Twenty-First International Conference on Intelligent User Interfaces, pp. 28-31. ACM, Sonoma (2016)Google Scholar
  9. 9.
    Aamodt, A.: Explanation-driven case-based reasoning. In: Wess, S., Richter, M., Althoff, K.-D. (eds.) EWCBR 1993. LNCS, vol. 837, pp. 274-288. Springer, Heidelberg (1994). doi: 10.1007/3-540-58330-0_93 CrossRefGoogle Scholar
  10. 10.
    Molineaux, M., Kuter, U., Klenk, M.: Discover history: understanding the past in planning and execution. In: Proceedings of the Eleventh International Conference on Autonomous Agents and Multi-agent Systems, pp. 989-996. IFAAMAS, Valencia (2012)Google Scholar
  11. 11.
    Leake, D.B.: CBR in context: the present and future. In: Leake, D.B. (ed.) Case-Based Reasoning: Experiences, Lessons, and Future Directions. AAAI Press/MIT Press, Menlo Park (1996)Google Scholar
  12. 12.
    Cunningham, P., Doyle, D., Loughrey, J.: An evaluation of the usefulness of case-based explanation. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 122-130. Springer, Heidelberg (2003). doi: 10.1007/3-540-45006-8_12 CrossRefGoogle Scholar
  13. 13.
    Sørmo, F., Cassens, J., Aamodt, A.: Explanation in case-based reasoning-perspectives and goals. Artif. Intell. Rev. 24(2), 109-143 (2005)CrossRefzbMATHGoogle Scholar
  14. 14.
    Roth-Berghofer, T.R.: Explanations and case-based reasoning: foundational issues. In: Funk, P., González Calero, P.A. (eds.) ECCBR 2004. LNCS (LNAI), vol. 3155, pp. 389-403. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-28631-8_29 CrossRefGoogle Scholar
  15. 15.
    Kofod-Petersen, A., Cassens, J.: Explanations and context in ambient intelligent systems. In: Kokinov, B., Richardson, D.C., Roth-Berghofer, T.R., Vieu, L. (eds.) CONTEXT 2007. LNCS (LNAI), vol. 4635, pp. 303-316. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-28631-8_29 CrossRefGoogle Scholar
  16. 16.
    Brüninghaus, S., Ashley, K.D.: Combining case-based and model-based reasoning for predicting the outcome of legal cases. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 65-79. Springer, Heidelberg (2003). doi: 10.1007/3-540-45006-8_8 CrossRefGoogle Scholar
  17. 17.
    Massie, S., Craw, S., Wiratunga, N.: Visualisation of case-case reasoning for explanation. In: Proceedings of the Seventh European Conference on Case-Based Reasoning Workshops, pp. 135-144. Madrid, Spain (2004)Google Scholar
  18. 18.
    Tintarev, N., Masthoff, J.: A survey of explanations in recommender systems. In: Proceedings of the Twenty-Third International Conference on Data Engineering Workshops, pp. 801-810. IEEE, Istanbul (2007)Google Scholar
  19. 19.
    McSherry, D.: Explanation in recommender systems. Artif. Intell. Rev. 24(2), 179-197 (2005)CrossRefzbMATHGoogle Scholar
  20. 20.
    Muhammad, K., Lawlor, A., Rafter, R., Smyth, B.: Great explanations: opinionated explanations for recommendations. In: Muhammad, K., Lawlor, A., Rafter, R., Smyth, B. (eds.) ICCBR 2015. LNCS, vol. 9343, pp. 244-258. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24586-7_17 CrossRefGoogle Scholar
  21. 21.
    Tavakolifard, M., Herrmann, P., Öztürk, P.: Analogical trust reasoning. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (eds.) IFIPTM 2009. IFIP AICT, vol. 300, pp. 149-163. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-02056-8_10 CrossRefGoogle Scholar
  22. 22.
    Briggs, P., Smyth, B.: Provenance, trust, and sharing in peer-to-peer case-based web search. In: Althoff, K.-D., Bergmann, R., Minor, M., Hanft, A. (eds.) ECCBR 2008. LNCS, vol. 5239, pp. 89-103. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-85502-6_6 CrossRefGoogle Scholar
  23. 23.
    Leake, D.B., Whitehead, M.: Case provenance: The value of remembering case sources. In: Weber, R.O., Richter, M.M. (eds.) ICCBR 2007. LNCS, vol. 4626, pp. 194-208. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74141-1_14 CrossRefGoogle Scholar
  24. 24.
    Kaniarasu, P., Steinfeld, A., Desai, M., Yanco, H.A.: Robot confidence and trust alignment. In: Proceedings of the Eighth International Conference on Human-Robot Interaction, pp. 155-156. ACM, Tokyo (2013)Google Scholar
  25. 25.
    Saleh, J.A., Karray, F., Morckos, M.: Modelling of robot attention demand in human-robot interaction using finite fuzzy state automata. In: Proceedings of the International Conference on Fuzzy Systems, pp. 1-8. IEEE, Brisbane (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Knexus Research CorporationSpringfieldUSA
  2. 2.Navy Center for Applied Research in AINaval Research Laboratory (Code 5514)Washington, DCUSA

Personalised recommendations