XAI—An Approach for Understanding Decisions Made by Neural Network

Abstract

For the longest time, most machine learning models have been viewed as black boxes. However, decisions with high stake cannot be taken without an explanation. So, model explainability and interpretability have become very important. Explainable artificial intelligence is a novel approach that aims to address how a model makes decisions. This paper throws light on understanding contribution of layer wise neurons involved in making decisions. Explanation of correlation between inputs and outputs and how the model arrived at that decision is the main focus of this application. The proposed application provides an explanation in the form of probabilities for numeric attributes and superpixel shading for images. A lot of models can be passed to this application and hence it makes it useful in all the use cases where neural networks are used for decision making. For the experimental purpose, we have considered a model for detecting the presence of malaria parasite in the blood sample image and classify it as parasitized or uninfected class. In the results section, different input images are taken and the analysis about the factors contributing to the model output is discussed.