Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Logic, Rationality and Interaction

LORI 2021: Logic, Rationality, and Interaction pp 232–245Cite as

Modelling Accuracy and Trustworthiness of Explaining Agents

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 13039)

Abstract

Current research in Explainable AI includes post-hoc explanation methods that focus on building transparent explaining agents able to emulate opaque ones. Such agents are naturally required to be accurate and trustworthy. However, what it means for an explaining agent to be accurate and trustworthy is far from being clear. We characterize accuracy and trustworthiness as measures of the distance between the formal properties of a given opaque system and those of its transparent explanantes. To this aim, we extend Probabilistic Computation Tree Logic with operators to specify degrees of accuracy and trustworthiness of explaining agents. We also provide a semantics for this logic, based on a multi-agent structure and relative model-checking algorithms. The paper concludes with a simple example of a possible application.

Keywords

  • Explainable AI
  • Accuracy
  • Trustworthiness

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In particular, \(\sigma (\varPi )\) is usually the \(\sigma \)-algebra generated by the cylinder sets of \(\varPi \) that allows \(\sigma (\varPi )\) to be always a measurable space (see [1, 9]).

  2. 2.

    As in standard PCTL, the CTL existential and universal quantifiers, expressing quantification over paths satisfying a given formula \(\psi \), here are omitted. It is easy to prove that they correspond to special cases of probabilistic quantification. In particular, \(\exists \psi \iff P_{>0}\psi \) and \(\forall \psi \iff P_{=1}\psi \). For the details, see [1].

  3. 3.

    Notice that path-formulas \(\psi \) are usually not considered in a typical probabilistic model-checking workflow. For the details of the procedure, see [1].

  4. 4.

    Notice that, this must not be intended as a conditional probability.

  5. 5.

    Here, minimality is defined as for Eq. (2).

  6. 6.

    This represents a typical example of a stochastic machine learning model. According to the classification we propose in the introduction, it can be classified as an opaque but comprehensible model. For more details, see [2].

  7. 7.

    Remeber that an explanans is an agent able to (locally) emulate the behaviour of the target-system and usually consider more transparent than this one.

  8. 8.

    A Python implementation of Algorithm 1 is available at https://github.com/dasaro/ATCTL together with details on how to reproduce the results from the example.

References

  1. Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)

    Google Scholar 

  2. Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, 5th edn. Springer, New York (2007). https://www.worldcat.org/oclc/71008143

  3. Chen, T., Primiero, G., Raimondi, F., Rungta, N.: A computationally grounded, weighted doxastic logic. Studia Logica 104(4), 679–703 (2016)

    CrossRef  Google Scholar 

  4. D’Asaro, F.A., Primiero, G.: Probabilistic typed natural deduction for trustworthy computations. In: Proceedings of the 22nd International Workshop on Trust in Agent Societies (TRUST2021 @ AAMAS) (2021)

    Google Scholar 

  5. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 93:1–93:42 (2019). https://doi.org/10.1145/3236009

  6. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects Comput. 6(5), 512–535 (1994)

    CrossRef  Google Scholar 

  7. Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy 23(1), 18 (2021). https://doi.org/10.3390/e23010018

    CrossRef  Google Scholar 

  8. Lomuscio, A., Raimondi, F.: mcmas: a model checker for multi-agent systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 450–454. Springer, Heidelberg (2006). https://doi.org/10.1007/11691372_31

    CrossRef  Google Scholar 

  9. Revuz, D.: Markov Chains. Elsevier, New York (2008)

    Google Scholar 

  10. Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., Zhong, C.: Interpretable machine learning: Fundamental principles and 10 grand challenges. CoRR abs/2103.11251 (2021). https://arxiv.org/abs/2103.11251

Download references

Acknowledgments

This research has been funded by the Department of Philosophy “Piero Martinetti” of the University of Milan under the Project “Departments of Excellence 2018–2022” awarded by the Ministry of Education, University and Research (MIUR). The authors also thankfully acknowledge the support of the Italian Ministry of University and Research (PRIN 2017 project n. 20173YP4N3).

Author information

Authors and Affiliations

  1. University of Milan, Milan, Italy

    Alberto Termine, Giuseppe Primiero & Fabio Aurelio D’Asaro

Authors
  1. Alberto Termine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Primiero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabio Aurelio D’Asaro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Termine .

Editor information

Editors and Affiliations

  1. Indian Statistical Institute, Chennai, Tamil Nadu, India

    Sujata Ghosh

  2. Department of Philosophy, Stanford University, Stanford, CA, USA

    Thomas Icard

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Termine, A., Primiero, G., D’Asaro, F.A. (2021). Modelling Accuracy and Trustworthiness of Explaining Agents. In: Ghosh, S., Icard, T. (eds) Logic, Rationality, and Interaction. LORI 2021. Lecture Notes in Computer Science(), vol 13039. Springer, Cham. https://doi.org/10.1007/978-3-030-88708-7_19

Download citation

search

Navigation