Cognitive Reasoning and Trust in Human-Robot Interactions

  • Marta Kwiatkowska
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10185)


We are witnessing accelerating technological advances in autonomous systems, of which driverless cars and home-assistive robots are prominent examples. As mobile autonomy becomes embedded in our society, we increasingly often depend on decisions made by mobile autonomous robots and interact with them socially. Key questions that need to be asked are how to ensure safety and trust in such interactions. How do we know when to trust a robot? How much should we trust? And how much should the robots trust us? This paper will give an overview of a probabilistic logic for expressing trust between human or robotic agents such as “agent A has 99% trust in agent B’s ability or willingness to perform a task” and the role it can play in explaining trust-based decisions and agent’s dependence on one another. The logic is founded on a probabilistic notion of belief, supports cognitive reasoning about goals and intentions, and admits quantitative verification via model checking, which can be used to evaluate trust in human-robot interactions. The paper concludes by summarising future challenges for modelling and verification in this important field.


  1. 1.
    Bianco, A., de Alfaro, L.: Model checking of probabalistic and nondeterministic systems. FSTTCS 1995, 499–513 (1995)zbMATHGoogle Scholar
  2. 2.
    Bratman, M.E.: Intentions, Plans, and Practical Reason. Harvard University Press, Massachusetts (1987)Google Scholar
  3. 3.
    Conitzer, V., Sinnott-Armstrong, W., Borg, J.S., Deng, Y., Kramer, M.: Moral decision making frameworks for Artificial Intelligence. In: AAAI 2017 (2017, to appear)Google Scholar
  4. 4.
    Falcone, R., Castelfranchi, C.: Social trust: a cognitive approach. In: Trust and Deception in Virtual Societies, pp. 55–90. Kluwer (2001)Google Scholar
  5. 5.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Form. Aspects Comput. 6(5), 512–535 (1994)CrossRefzbMATHGoogle Scholar
  6. 6.
    Hardin, R.: Trust and Trustworthiness. Russell Sage Foundation (2002)Google Scholar
  7. 7.
    Huang, X., Kwiatkowska, M.: Reasoning about cognitive trust in stochastic multiagent systems. In: AAAI 2017 (2017, to appear)Google Scholar
  8. 8.
    Kuipers, B.: What is trust and how can my robot get some? (presentation). In: RSS 2016 Workshop on Social Trust in Autonomous Robots (2016)Google Scholar
  9. 9.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22110-1_47 CrossRefGoogle Scholar
  10. 10.
    Lahijanian, M., Kwiatkowska, M.: Social trust: a major challenge for the future of autonomous systems. In: AAAI Fall Symposium on Cross-Disciplinary Challenges for Autonomous Systems, AAAI Fall Symposium. AAAI, AAAI Press (2016)Google Scholar
  11. 11.
    Lee, D.: US opens investigation into Tesla after fatal crash. British Broadcasting Corporation (BBC) News, 1 July 2016.
  12. 12.
    Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors J. Hum. Factors Ergon. Soc. 46(1), 50–80 (2004)Google Scholar
  13. 13.
    Letchford, J., Conitzer, V., Jain, K.: An “ethical” game-theoretic solution concept for two-player perfect-information games. In: Proceedings of the Fourth Workshop on Internet and Network Economics (WINE-08), pp. 696–707 (2008)Google Scholar
  14. 14.
    Mayer, R.C., Davis, J.H., Schoorman, F.D.: An integrative model of organizational trust. Acad. Manage. Rev. 20(3), 709–734 (1995)Google Scholar
  15. 15.
    McKnight, D.H., Choudhury, V., Kacmar, C.: Developing and validating trust measures for e-commerce: an integrative typology. Inf. Syst. Res. 13(3), 334–359 (2002)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of OxfordOxfordUK

Personalised recommendations