A Case Based Deep Neural Network Interpretability Framework and Its User Study
Despite its popularity, the decision making process of a Deep Neural Network (DNN) model is opaque to users, making it difficult to understand the behaviour of the model. We present the design of a Web-based DNN interpretability framework which is based on the core notions in case-based reasoning approaches where exemplars (e.g., data points considered similar to a chosen data point) are utilised to help achieve effective interpretation. We demonstrate the framework via a Web based tool called Deep Explorer (DeX) and present the results of user acceptance studies. Our studies showed the effectiveness of the tool in gaining a better understanding of the decision making process of a DNN model as well as the efficacy of the case-based approach in improving DNN interpretability.
KeywordsDeep neural network interpretability Visualisation Decision boundaries Interpretable machine learning
The authors thank all participants who took part in the application user study.
- 2.Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)Google Scholar
- 4.Kim, B., Khanna, R., Koyejo, O.O.: Examples are not enough, learn to criticize! Criticism for interpretability. In: Advances in Neural Information Processing Systems, pp. 2280–2288 (2016)Google Scholar
- 6.Fong, R.C., Vedaldi, A.: Interpretable explanations of black boxes by meaningful perturbation. In: IEEE Conference on Computer Vision, pp. 3449–3457 (2017)Google Scholar
- 7.Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. In: The 34th International Conference on Machine Learning, pp. 1885–1894 (2017)Google Scholar
- 8.Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015)
- 9.Wu, H., Wang, C., Yin, J., Lu, K., Zhu, L.: Sharing deep neural network models with interpretation. In: Conference on World Wide Web, pp. 177–186 (2018)Google Scholar
- 12.Andoni, A., Indyk, P., Laarhoven, T., Razenshteyn, I., Schmidt, L.: Practical and optimal LSH for angular distance. In: Advances in Neural Information Processing Systems, pp. 1225–1233 (2015)Google Scholar
- 13.Brooke, J.: Sus-a quick and dirty usability scale. In: Usability Evaluation in Industry, vol. 189, no. 194, pp. 4–7 (1996)Google Scholar