Comparing Strategies for Post-Hoc Explanations in Machine Learning Models

Abstract

Most of the machine learning models act as black boxes, and hence, the need for interpreting them is rising. There are multiple approaches to understand the outcomes of a model. But in order to be able to trust the interpretations, there is a need to have a closer look at these approaches. This project compared three such frameworks—ELI5, LIME and SHAP. ELI5 and LIME follow the same approach toward interpreting the outcomes of machine learning algorithms by building an explainable model in the vicinity of the datapoint that needs to be explained, whereas SHAP works with Shapley values, a game theory approach toward assigning feature attribution. LIME outputs an R-squared value along with its feature attribution reports which help in quantifying the trust one must have in those interpretations. The R-squared value for surrogate models within different machine learning models varies. SHAP trades-off accuracy with time (theoretically). Assigning SHAP values to features is a time and computationally consuming task, and hence, it might require sampling beforehand. SHAP triumphs over LIME with respect to optimization of different kinds of machine learning models as it has explainers for different types of machine learning models, and LIME has one generic explainer for all model types.