A Case-Based Explanation System for Black-Box Systems Article DOI:
Cite this article as: Nugent, C. & Cunningham, P. Artif Intell Rev (2005) 24: 163. doi:10.1007/s10462-005-4609-5 Abstract
Most users of machine-learning products are reluctant to use them without any sense of the underlying logic that has led to the system’s predictions. Unfortunately many of these systems lack any transparency in the way they operate and are deemed to be black boxes. In this paper we present a Case-Based Reasoning (CBR) solution to providing supporting explanations of black-box systems. This CBR solution has two key facets; it uses local information to assess the importance of each feature and using this, it selects the cases from the data used to build the black-box system for use in explanation. The retrieval mechanism takes advantage of the derived feature importance information to help select cases that are a better reflection of the black-box solution and thus more convincing explanations.
Keywords black-box systems case-based reasoning explanation References Andrews, R., Diederich, J., Tickle, A. 1995 A Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks Knowledge Based Systems 8 373 389 CrossRef Google Scholar Atkeson, C.G., Moore, A.W., Schaal, S. 1997 Locally Weighted Learning Artificial Intelligence Review 11 11 73 CrossRef Google Scholar
Cassens, J. (2004). Knowing What to Explain and When In Gervás, P., Gupta, K. (eds.) Proceedings of the ECCBR 2004 Workshops. 97–104, Technical Report 142-04, Departmento de Sistemas Informáticos y Programación, Universidad Complutense de Madrid, Madrid, Spain
Cunningham, P., Doyle, D., Loughrey, J. (2003). An Evaluation of the Usefulness of Case-Based Explanation In Ashley, K. D., Bridge, D. G. (eds.) ICCBR, Vol. 2689 of Lecture Notes in Computer Science. 122–130, Springer: Berlin
Domingos, P. 1998 Knowledge Discovery Via Multiple Models Intelligent Data Analysis 2 187 202 CrossRef Google Scholar Doyle, D., Cunningham, P., Bridge, D., Rahman, Y. 2004 Explanation Oriented Retrieval Funk, P. Calero, P. eds. Advances in Case-Based Reasoning (Procs. of the Seventh European Conference on Case-Based Reasoning) Springer Berlin 157 168 Google Scholar Gentner, D., Loewenstein, J., Thompson, L. 2003 Learning and Transfer: A General Role for Analogical Encoding Journal of Educational Psychology 95 393 408 CrossRef Google Scholar Leake, D. 1996 CBR in Context: The Present and Future Leake, D. eds. Case-Based Reasoning: Experiences, Lessons, and Future Directions AAAI Press Menlo Park, CA 3 30 Google Scholar Majchrzak, A., Gasser, L. 1991 On using Artificial Intelligence to Integrate the Design of Organizational and Process Change in US Manufacturing AI and Society 5 321 338 CrossRef Google Scholar
McSherry, D. (2003). Explanation in Case-based Reasoning: an Evidential Approach In Proceeding of 8th UK Workshop on Case-Based Reasoning. 47–55.
McSherry, D. 2004 Explaining the Pros and Cons of Conclusions in CBR Funk, P. Calero, P. eds. Advances in Case-Based Reasoning (Proceedings of the Seventh European Conference on Case-Based Reasoning) Springer Berlin 317 330 Google Scholar
Sørmo, F., Cassens, J. (2004). Explanation Goals in Case-Based Reasoning. In Gervás, P., Gupta, K. (eds.) Proceedings of the ECCBR 2004 Workshops. 165–174, Technical Report 142-04, Departmento de Sistemas Informáticos y Programación, Universidad Complutense de Madrid, Madrid, Spain.
Sórmo, F., Cassens, J., Aamodt, A. (2005). Explanation in Case-Based Reasoning: Perspectives and Goals Artificial Intelligence Review This Issue.
Tickle, A., Andrews, R., Golea, R., Diederich, J. 1998 The Truth Will Come to Light: Directions and Challenges in Extracting Rules from Trained Neural Networks IEEE Transactions on Neural Networks 9 1057 1068 CrossRef Google Scholar Walsh, P., Cunningham, P., Rothenberg, S.J., O’Doherty, S., Hoey, H., Healy, R. 2004 An Artificial Neural Network Ensemble to Predict Disposition and Length of Stay in Children Presenting with Bronchiolitis European Journal of Emergency Medicine 11 259 264 CrossRef PubMed Google Scholar Zhou, Z.-H., Jiang, Y. 2003 Diagnosis with C4.5 Preceded by Artificial Neural Network Ensemble Transactions on Information Technology in Biomedicine 7 37 42 CrossRef PubMed Google Scholar