Meaningful Data Sampling for a Faithful Local Explanation Method

  • Peyman RasouliEmail author
  • Ingrid Chieh YuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


Data sampling has an important role in the majority of local explanation methods. Generating neighborhood samples using either the Gaussian distribution or the distribution of training data is a widely-used procedure in the tabular data case. Generally, this approach has several weaknesses: first, it produces a uniform data which may not represent the actual distribution of samples; second, disregarding the interaction between features tends to create unlikely samples; and third, it may fail to define a compact and diverse locality for the sample being explained. In this paper, we propose a sampling methodology based on observation-level feature importance to derive more meaningful perturbed samples. To evaluate the efficiency of the proposed approach we applied it to the LIME explanation method. The conducted experiments demonstrate considerable improvements in terms of fidelity and explainability.


  1. 1.
    Biau, G., Scornet, E.: A random forest guided tour. Test 25(2), 197–227 (2016)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)CrossRefGoogle Scholar
  3. 3.
    Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: Proceedings Machine Learning, pp. 194–202 (1995)CrossRefGoogle Scholar
  4. 4.
    Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., Giannotti, F.: Local rule-based explanations of black box decision systems. CoRR (2018)Google Scholar
  5. 5.
    Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 93 (2018)Google Scholar
  6. 6.
    Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques. Elsevier Science, Amsterdam (2011)zbMATHGoogle Scholar
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105. Curran Associates Inc. (2012)Google Scholar
  8. 8.
    Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, pp. 4765–4774 (2017)Google Scholar
  9. 9.
    Molnar, C.: Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (2019).
  10. 10.
    Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016)Google Scholar
  11. 11.
    Robnik-Šikonja, M., Bohanec, M.: Perturbation-based explanations of prediction models. In: Zhou, J., Chen, F. (eds.) Human and Machine Learning. HIS, pp. 159–175. Springer, Cham (2018). Scholar
  12. 12.
    Robnik-Sikonja, M., Kononenko, I.: Explaining classifications for individual instances. IEEE TKDE 20(5), 589–600 (2008)Google Scholar
  13. 13.
    Saabas, A.: TreeInterpreter Library (2019).
  14. 14.
    Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: Proceedings of the 34th International Conference on Machine Learning, pp. 3145–3153 (2017)Google Scholar
  15. 15.
    Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014)CrossRefGoogle Scholar
  16. 16.
    Štrumbelj, E., Kononenko, I., Robnik Šikonja, M.: Explaining instance classifications with interactions of subsets of feature values. Data Knowl. Eng. 68(10), 886–904 (2009)CrossRefGoogle Scholar
  17. 17.
    Sturges, H.A.: The choice of a class interval. J. Am. Stat. Assoc. 21(153), 65–66 (1926)CrossRefGoogle Scholar
  18. 18.
    Tipping, M.E., Bishop, C.M.: Probabilistic principal component analysis. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 61(3), 611–622 (1999) MathSciNetCrossRefGoogle Scholar
  19. 19.
    Turner, R.: A model explanation system. In: IEEE 26th International Workshop on Machine Learning for Signal Processing, pp. 1–6 (2016)Google Scholar
  20. 20.
    Zhu, J., Zou, H., Rosset, S., Hastie, T.: Multi-class AdaBoost*. Stat. Interface 2, 349–360 (2009)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of InformaticsUniversity of OsloOsloNorway

Personalised recommendations