Abstract
As we deploy autonomous agents in safety-critical domains, it becomes important to develop an understanding of their internal mechanisms and representations. We outline an approach to imitation learning for reverse-engineering black box agent policies in MDP environments, yielding simplified, interpretable models in the form of decision trees. As part of this process, we explicitly model and learn agents’ latent state representations by selecting from a large space of candidate features constructed from the Markov state. We present initial promising results from an implementation in a multi-agent traffic environment.
Keywords
- Explainable artificial intelligence
- Interpretability
- Imitation learning
- Representation learning
- Decision tree
- Traffic modelling
Supported by an EPSRC/Thales industrial CASE award in autonomous systems.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Icons from users Freepik and Pixel Perfect at www.flaticon.com.
References
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth & Brooks, Cole Statistics/Probability Series, California (1984)
Coppens, Y., Efthymiadis, K., Lenaerts, T., Nowé, A., Miller, T., Weber, R., Magazzeni, D.: Distilling deep reinforcement learning policies in soft decision trees. In: Proceedings of the IJCAI 2019 Workshop on Explainable AI, pp. 1–6 (2019)
Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., Turini, F.: Factual and counterfactual explanations for black box decision making. IEEE Intell. Syst. (2019)
Lesort, T., Díaz-Rodríguez, N., Goudou, J.F., Filliat, D.: State representation learning for control: an overview. Neural Networks 108, 379–392 (2018)
Samek, W., Wiegand, T., Müller, K.R.: Explainable AI: understanding, visualizing and interpreting deep learning models. arXiv:1708.08296 (2017)
Turnbull, O., Lawry, J., Lowenberg, M., Richards, A.: A cloned linguistic decision tree controller for real-time path planning in hostile environments. Fuzzy Sets Syst. 293, 1–29 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bewley, T., Lawry, J., Richards, A. (2021). Modelling Agent Policies with Interpretable Imitation Learning. In: Heintz, F., Milano, M., O'Sullivan, B. (eds) Trustworthy AI - Integrating Learning, Optimization and Reasoning. TAILOR 2020. Lecture Notes in Computer Science(), vol 12641. Springer, Cham. https://doi.org/10.1007/978-3-030-73959-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-73959-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73958-4
Online ISBN: 978-3-030-73959-1
eBook Packages: Computer ScienceComputer Science (R0)