Skip to main content
SpringerLink
Log in

Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges

  • Conference paper
  • First Online:
ECML PKDD 2020 Workshops (ECML PKDD 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1323))

Abstract

We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This project is funded by the Bavarian State Ministry of Science and the Arts and coordinated by the Bavarian Research Institute for Digital Transformation (bidt) and supported by the German Federal Ministry of Education and Research (BMBF) under Grant No. 01IS18036A. The authors of this work take full responsibilities for its content.

  2. 2.

    Sometimes the term Explainable AI is used.

  3. 3.

    The random forest paper has been cited over 60,000 times (Google Scholar; September 2020) and there are many papers improving the importance measure ([44, 55, 110, 111]) which are also cited frequently.

  4. 4.

    Not to be confused with the research field of sensitivity analysis, which studies the uncertainty of outputs in mathematical models and systems. There are methodological overlaps (e.g., Shapley values), but also differences in methods and how input data distributions are handled.

  5. 5.

    Some surveys distinguish between ante-hoc (or transparent design, white-box models, inherently interpretable model) and post-hoc IML method, depending on whether interpretability is considered at model design and training or after training, leaving the (black-box) model unchanged. Another category separates model-agnostic and model-specific methods.

  6. 6.

    This blurs the line between an “inherently interpretable” and a “black-box” model.

  7. 7.

    Surrogate models are related to knowledge distillation and the teacher-student model.

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: Advances in Neural Information Processing Systems, pp. 9505–9515 (2018)

    Google Scholar 

  3. Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Parzen, E., Tanabe, K., Kitagawa, G. (eds.) Selected Papers of Hirotugu Akaike, pp. 199–213. Springer, New york (1998). https://doi.org/10.1007/978-1-4612-1694-0_15

    Chapter  Google Scholar 

  4. Altmann, A., Toloşi, L., Sander, O., Lengauer, T.: Permutation importance: a corrected feature importance measure. Bioinformatics 26(10), 1340–1347 (2010)

    Article  Google Scholar 

  5. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst. 8(6), 373–389 (1995)

    Article  MATH  Google Scholar 

  6. Anjomshoae, S., Najjar, A., Calvaresi, D., Främling, K.: Explainable agents and robots: results from a systematic literature review. In: 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, 13–17 May 2019, pp. 1078–1088. International Foundation for Autonomous Agents and Multiagent Systems (2019)

    Google Scholar 

  7. Apley, D.W., Zhu, J.: Visualizing the effects of predictor variables in black box supervised learning models. arXiv preprint arXiv:1612.08468 (2016)

  8. Arya, V., Bellamy, R.K., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S.C., Houde, S., Liao, Q.V., Luss, R., Mojsilovic, A., et al.: AI explainability 360: an extensible toolkit for understanding data and machine learning models. J. Mach. Learn. Res. 21(130), 1–6 (2020)

    MATH  Google Scholar 

  9. Augasta, M.G., Kathirvalavakumar, T.: Rule extraction from neural networks–a comparative study. In: International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012), pp. 404–408. IEEE (2012)

    Google Scholar 

  10. Bastani, O., Kim, C., Bastani, H.: Interpreting blackbox models via model extraction. arXiv preprint arXiv:1705.08504 (2017)

  11. Biecek, P.: DALEX: explainers for complex predictive models in R. J. Mach. Learn. Res. 19(1), 3245–3249 (2018)

    MATH  Google Scholar 

  12. Botari, T., Hvilshøj, F., Izbicki, R., de Carvalho, A.C.: MeLIME: meaningful local explanation for machine learning models. arXiv preprint arXiv:2009.05818 (2020)

  13. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  14. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N.: Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1721–1730 (2015)

    Google Scholar 

  15. Carvalho, D.V., Pereira, E.M., Cardoso, J.S.: Machine learning interpretability: a survey on methods and metrics. Electronics 8(8), 832 (2019)

    Article  Google Scholar 

  16. Casalicchio, G., Molnar, C., Bischl, B.: Visualizing the feature importance for black box models. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 655–670. Springer (2018). https://doi.org/10.1007/978-3-030-10925-7_40

  17. Chromik, M., Schuessler, M.: A taxonomy for human subject evaluation of black-box explanations in XAI. In: ExSS-ATEC@ IUI (2020)

    Google Scholar 

  18. Craven, M., Shavlik, J.W.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, pp. 24–30 (1996)

    Google Scholar 

  19. Cutler, D.R., Edwards Jr., T.C., Beard, K.H., Cutler, A., Hess, K.T., Gibson, J., Lawler, J.J.: Random forests for classification in ecology. Ecology 88(11), 2783–2792 (2007)

    Article  Google Scholar 

  20. Dandl, S., Molnar, C., Binder, M., Bischl, B.: Multi-objective counterfactual explanations. arXiv preprint arXiv:2004.11165 (2020)

  21. Dhurandhar, A., Iyengar, V., Luss, R., Shanmugam, K.: TIP: typifying the interpretability of procedures. arXiv preprint arXiv:1706.02952 (2017)

  22. Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)

  23. Du, M., Liu, N., Hu, X.: Techniques for interpretable machine learning. Commun. ACM 63(1), 68–77 (2019)

    Article  Google Scholar 

  24. Fabi, K., Schneider, J.: On feature relevance uncertainty: a Monte Carlo dropout sampling approach. arXiv preprint arXiv:2008.01468 (2020)

  25. Fahrmeir, L., Tutz, G.: Multivariate Statistical Modelling Based on Generalized Linear Models. Springer, Cham (2013)

    MATH  Google Scholar 

  26. Fasiolo, M., Nedellec, R., Goude, Y., Wood, S.N.: Scalable visualization methods for modern generalized additive models. J. Comput. Graph. Stat. 29(1), 78–86 (2020)

    Article  MathSciNet  Google Scholar 

  27. Fasiolo, M., Wood, S.N., Zaffran, M., Nedellec, R., Goude, Y.: Fast calibrated additive quantile regression. J. Am. Stat. Assoc. 1–11 (2020)

    Google Scholar 

  28. Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res. 20(177), 1–81 (2019)

    MathSciNet  MATH  Google Scholar 

  29. Freiesleben, T.: Counterfactual explanations & adversarial examples-common grounds, essential differences, and potential transfers. arXiv preprint arXiv:2009.05487 (2020)

  30. Freitas, A.A.: Comprehensible classification models: a position paper. ACM SIGKDD Explor. Newslett. 15(1), 1–10 (2014)

    Article  Google Scholar 

  31. Friedler, S.A., Roy, C.D., Scheidegger, C., Slack, D.: Assessing the local interpretability of machine learning models. arXiv preprint arXiv:1902.03501 (2019)

  32. Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 1189–1232 (2001)

    Google Scholar 

  33. Friedman, J.H., Popescu, B.E., et al.: Predictive learning via rule ensembles. Ann. Appl. Stat. 2(3), 916–954 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Frosst, N., Hinton, G.: Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784 (2017)

  35. Fürnkranz, J., Gamberger, D., Lavrač, N.: Foundations of Rule Learning. Springer, Cham (2012)

    Book  MATH  Google Scholar 

  36. Gade, K., Geyik, S.C., Kenthapadi, K., Mithal, V., Taly, A.: Explainable AI in industry. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3203–3204 (2019)

    Google Scholar 

  37. Gauss, C.F.: Theoria motus corporum coelestium in sectionibus conicis solem ambientium, vol. 7. Perthes et Besser (1809)

    Google Scholar 

  38. Gelman, A., Hill, J.: Data Analysis Using Regression and Multilevel/hierarchical Models. Cambridge University Press, Cambridge (2006)

    Book  Google Scholar 

  39. Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E.: Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J. Comput. Graph. Stat. 24(1), 44–65 (2015)

    Article  MathSciNet  Google Scholar 

  40. Greenwell, B.M., Boehmke, B.C., McCarthy, A.J.: A simple and effective model-based variable importance measure. arXiv preprint arXiv:1805.04755 (2018)

  41. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. (CSUR) 51(5), 1–42 (2018)

    Article  Google Scholar 

  42. Hall, M., et al.: A systematic method to understand requirements for explainable AI(XAI) systems. In: Proceedings of the IJCAI Workshop on eXplainable Artificial Intelligence (XAI 2019), Macau, China (2019)

    Google Scholar 

  43. Hall, P., Gill, N., Kurka, M., Phan, W.: Machine learning interpretability with H2O driverless AI. H2O. AI (2017). http://docs.h2o.ai/driverless-ai/latest-stable/docs/booklets/MLIBooklet.pdf

  44. Hapfelmeier, A., Hothorn, T., Ulm, K., Strobl, C.: A new variable importance measure for random forests with missing data. Stat. Comput. 24(1), 21–34 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  45. Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models, vol. 43. CRC Press, Boca Raton (1990)

    MATH  Google Scholar 

  46. Hauenstein, S., Wood, S.N., Dormann, C.F.: Computing AIC for black-box models using generalized degrees of freedom: a comparison with cross-validation. Commun. Stat.-Simul. Comput. 47(5), 1382–1396 (2018)

    Article  MathSciNet  Google Scholar 

  47. Haunschmid, V., Manilow, E., Widmer, G.: audioLIME: listenable explanations using source separation. arXiv preprint arXiv:2008.00582 (2020)

  48. Head, M.L., Holman, L., Lanfear, R., Kahn, A.T., Jennions, M.D.: The extent and consequences of p-hacking in science. PLoS Biol. 13(3), e1002106 (2015)

    Article  Google Scholar 

  49. Hoffman, R.R., Mueller, S.T., Klein, G., Litman, J.: Metrics for explainable AI: challenges and prospects. arXiv preprint arXiv:1812.04608 (2018)

  50. Hooker, G.: Generalized functional ANOVA diagnostics for high-dimensional functions of dependent variables. J. Comput. Graph. Stat. 16(3), 709–732 (2007)

    Article  MathSciNet  Google Scholar 

  51. Hooker, G., Mentch, L.: Please stop permuting features: an explanation and alternatives. arXiv preprint arXiv:1905.03151 (2019)

  52. T. Hothorn, K. Hornik, and A. Zeileis. ctree: Conditional inference trees. The Comprehensive R Archive Network, 8, 2015

    Google Scholar 

  53. Hu, L., Chen, J., Nair, V.N., Sudjianto, A.: Locally interpretable models and effects based on supervised partitioning (LIME-SUP). arXiv preprint arXiv:1806.00663 (2018)

  54. Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., Baesens, B.: An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decis. Support Syst. 51(1), 141–154 (2011)

    Article  Google Scholar 

  55. Ishwaran, H., et al.: Variable importance in binary regression trees and forests. Electron. J. Stat. 1, 519–537 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  56. Ishwaran, H., Kogalur, U.B., Gorodeski, E.Z., Minn, A.J., Lauer, M.S.: High-dimensional variable selection for survival data. J. Am. Stat. Assoc. 105(489), 205–217 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  57. Janzing, D., Minorics, L., Blöbaum, P.: Feature relevance quantification in explainable AI: a causality problem. arXiv preprint arXiv:1910.13413 (2019)

  58. Klaise, J., Van Looveren, A., Vacanti, G., Coca, A.: Alibi: algorithms for monitoring and explaining machine learning models (2020). https://github.com/SeldonIO/alibi

  59. Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. arXiv preprint arXiv:1703.04730 (2017)

  60. König, G., Molnar, C., Bischl, B., Grosse-Wentrup, M.: Relative feature importance. arXiv preprint arXiv:2007.08283 (2020)

  61. Krishnan, S., Wu, E.: Palm: machine learning explanations for iterative debugging. In: Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, pp. 1–6 (2017)

    Google Scholar 

  62. Kumar, I.E., Venkatasubramanian, S., Scheidegger, C., Friedler, S.: Problems with Shapley-value-based explanations as feature importance measures. arXiv preprint arXiv:2002.11097 (2020)

  63. Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., Detyniecki, M.: The dangers of post-hoc interpretability: unjustified counterfactual explanations. arXiv preprint arXiv:1907.09294 (2019)

  64. Legendre, A.M.: Nouvelles méthodes pour la détermination des orbites des comètes. F. Didot (1805)

    Google Scholar 

  65. Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R.J., Wasserman, L.: Distribution-free predictive inference for regression. J. Am. Stat. Assoc. 113(523), 1094–1111 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  66. Letham, B., Rudin, C., McCormick, T.H., Madigan, D., et al.: Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. Ann. Appl. Stat. 9(3), 1350–1371 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  67. Lipton, Z.C.: The mythos of model interpretability. Queue 16(3), 31–57 (2018)

    Article  Google Scholar 

  68. Lundberg, S.M., Erion, G.G., Lee, S.-I.: Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888 (2018)

  69. Lundberg,S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, pp. 4765–4774 (2017)

    Google Scholar 

  70. Ma, S., Tourani, R.: Predictive and causal implications of using Shapley value for model interpretation. In: Proceedings of the 2020 KDD Workshop on Causal Discovery, pp. 23–38. PMLR (2020)

    Google Scholar 

  71. Miller, T.: Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267, 1–38 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  72. Ming, Y., Qu, H., Bertini, E.: Rulematrix: visualizing and understanding classifiers with rules. IEEE Trans. Vis. Comput. Graph. 25(1), 342–352 (2018)

    Article  Google Scholar 

  73. Mohseni, S., Ragan, E.D.: A human-grounded evaluation benchmark for local explanations of machine learning. arXiv preprint arXiv:1801.05075 (2018)

  74. Mohseni, S., Zarei, N., Ragan, E.D.: A multidisciplinary survey and framework for design and evaluation of explainable AI systems. arXiv, pages arXiv-1811 (2018)

    Google Scholar 

  75. Molnar, C.: Interpretable Machine Learning (2019). https://christophm.github.io/interpretable-ml-book/

  76. Molnar, C., Bischl, B., Casalicchio, G.: iml: an R package for interpretable machine learning. JOSS 3(26), 786 (2018)

    Article  Google Scholar 

  77. Molnar, C., Casalicchio, G., Bischl, B.: Quantifying model complexity via functional decomposition for better post-hoc interpretability. In: Cellier, P., Driessens, K. (eds.) ECML PKDD 2019. CCIS, vol. 1167, pp. 193–204. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43823-4_17

    Chapter  Google Scholar 

  78. Molnar, C., König, G., Bischl, B., Casalicchio, G.: Model-agnostic feature importance and effects with dependent features-a conditional subgroup approach. arXiv preprint arXiv:2006.04628 (2020)

  79. Molnar, C., et al.: Pitfalls to avoid when interpreting machine learning models. arXiv preprint arXiv:2007.04131 (2020)

  80. Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.-R.: Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recogn. 65, 211–222 (2017)

    Article  Google Scholar 

  81. Mothilal, R.K., Sharma, A., Tan, C.: Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 607–617 (2020)

    Google Scholar 

  82. Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Definitions, B.Y.: Methods, and applications in interpretable machine learning. Proc. Nat. Acad. Sci. 116(44), 22071–22080 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  83. Nori, H., Jenkins, S., Koch, P., Caruana, R.: Interpretml: a unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223 (2019)

  84. Olah, C., Mordvintsev, A., Schubert, L.: Feature visualization. Distill (2017). https://distill.pub/2017/feature-visualization

  85. Paluszynska, A., Biecek, P., Jiang, Y.: Random forest explainer: explaining and visualizing random forests in terms of variable importance, R package version 0.10.1 (2020)

    Google Scholar 

  86. Philipp, M., Rusch, T., Hornik, K., Strobl, C.: Measuring the stability of results from supervised statistical learning. J. Comput. Graph. Stat. 27(4), 685–700 (2018)

    Article  MathSciNet  Google Scholar 

  87. Poursabzi-Sangdeh, F., Goldstein, D.G., Hofman, J.M., Vaughan, J.W., Wallach, H.: Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018)

  88. Preece, A., Harborne, D., Braines, D., Tomsett, R., Chakraborty, S.: Stakeholders in explainable AI. arXiv preprint arXiv:1810.00184 (2018)

  89. Puri, N., Gupta, P., Agarwal, P., Verma, S., Krishnamurthy, B.: Magix: model agnostic globally interpretable explanations. arXiv preprint arXiv:1706.07160 (2017)

  90. Quetelet, L.A.J.: Recherches sur la population, les naissances, les décès, les prisons, les dépôts de mendicité, etc. dans le royaume des Pays-Bas (1827)

    Google Scholar 

  91. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2020)

    Google Scholar 

  92. Rabold, J., Deininger, H., Siebers, M., Schmid, U.: Enriching visual with verbal explanations for relational concepts – combining LIME with Aleph. In: Cellier, P., Driessens, K. (eds.) ECML PKDD 2019. CCIS, vol. 1167, pp. 180–192. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43823-4_16

    Chapter  Google Scholar 

  93. Rabold, J., Siebers, M., Schmid, U.: Explaining black-box classifiers with ILP – empowering LIME with Aleph to approximate non-linear decisions with relational rules. In: Riguzzi, F., Bellodi, E., Zese, R. (eds.) ILP 2018. LNCS (LNAI), vol. 11105, pp. 105–117. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99960-9_7

    Chapter  Google Scholar 

  94. Rahnama, A.H.A., Boström, H.: A study of data and label shift in the LIME framework. arXiv preprint arXiv:1910.14421 (2019)

  95. Ribeiro, M.T., Singh, S., Guestrin, C.: why should i trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016)

    Google Scholar 

  96. Rosenfeld, A., Richardson, A.: Explainability in human-agent systems. Auton. Agent. Multi-Agent Syst. 33(6), 673–705 (2019)

    Article  Google Scholar 

  97. Samek, W., Müller, K.-R.: Towards explainable artificial intelligence. In: Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Müller, K.-R. (eds.) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. LNCS (LNAI), vol. 11700, pp. 5–22. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-28954-6_1

    Chapter  Google Scholar 

  98. Santosa, F., Symes, W.W.: Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 7(4), 1307–1330 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  99. Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5(2), 197–227 (1990)

    Google Scholar 

  100. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  101. Schölkopf, B.: Causality for machine learning. arXiv preprint arXiv:1911.10500 (2019)

  102. Schwarz, G., et al.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  103. Shankaranarayana, S.M., Runje, D.: ALIME: autoencoder based approach for local interpretability. In: Yin, H., Camacho, D., Tino, P., Tallón-Ballesteros, A.J., Menezes, R., Allmendinger, R. (eds.) IDEAL 2019. LNCS, vol. 11871, pp. 454–463. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33607-3_49

    Chapter  Google Scholar 

  104. Shapley, L.S.: A value for N-person games. Contrib. Theory Games 2(28), 307–317 (1953)

    MathSciNet  MATH  Google Scholar 

  105. Shrikumar, A., Greenside, P., Shcherbina, A., Kundaje, A.: Not just a black box: learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713 (2016)

  106. Sill, J.: Monotonic networks. In: Advances in Neural Information Processing Systems, pp. 661–667 (1998)

    Google Scholar 

  107. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

  108. Starr, W.: Counterfactuals (2019)

    Google Scholar 

  109. Stigler, S.M.: The History of Statistics: The Measurement of Uncertainty Before 1900. Harvard University Press, Cambridge (1986)

    MATH  Google Scholar 

  110. Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., Zeileis, A.: Conditional variable importance for random forests. BMC Bioinf. 9(1), 307 (2008)

    Article  Google Scholar 

  111. Strobl, C., Boulesteix, A.-L., Zeileis, A., Hothorn, T.: Bias in random forest variable importance measures: illustrations, sources and a solution. BMC Bioinf. 8(1), 25 (2007)

    Article  Google Scholar 

  112. Štrumbelj, E., Kononenko, I.: Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41(3), 647–665 (2014)

    Article  Google Scholar 

  113. Sundararajan, M., Najmi, A.: The many Shapley values for model explanation. arXiv preprint arXiv:1908.08474 (2019)

  114. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365 (2017)

  115. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc.: Ser. B (Methodol.) 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  116. Tolomei, G., Silvestri, F., Haines, A., Lalmas, M.: Interpretable predictions of tree-based ensembles via actionable feature tweaking. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 465–474 (2017)

    Google Scholar 

  117. Ustun, B., Rudin, C.: Supersparse linear integer models for optimized medical scoring systems. Mach. Learn. 102(3), 349–391 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  118. Ustun, B., Spangher, A., Liu, Y.: Actionable recourse in linear classification. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 10–19 (2019)

    Google Scholar 

  119. Vapnik, V., Chervonenkis, A.: Theory of pattern recognition (1974)

    Google Scholar 

  120. Vilone, G., Longo, L.: Explainable artificial intelligence: a systematic review. arXiv preprint arXiv:2006.00093 (2020)

  121. Visani, G., Bagli, E., Chesani, F.: Optilime: Optimized LIME explanations for diagnostic computer algorithms. arXiv preprint arXiv:2006.05714 (2020)

  122. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv. JL Tech. 31, 841 (2017)

    Google Scholar 

  123. Wang, F., Rudin, C.: Falling rule lists. In: Artificial Intelligence and Statistics, pp. 1013–1022 (2015)

    Google Scholar 

  124. Watson, D.S., Wright, M.N.: Testing conditional independence in supervised learning algorithms. arXiv preprint arXiv:1901.09917 (2019)

  125. Wei, P., Lu, Z., Song, J.: Variable importance analysis: a comprehensive review. Reliabil. Eng. Syst. Saf. 142, 399–432 (2015)

    Article  Google Scholar 

  126. Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viégas, F., Wilson, J.: The what-if tool: interactive probing of machine learning models. IEEE Trans. Vis. Comput. Graph. 26(1), 56–65 (2019)

    Google Scholar 

  127. Williamson, B.D., Feng, J.: Efficient nonparametric statistical inference on population feature importance using Shapley values. arXiv preprint arXiv:2006.09481 (2020)

  128. Zeileis, A., Hothorn, T., Hornik, K.: Model-based recursive partitioning. J. Comput. Graph. Stat. 17(2), 492–514 (2008)

    Article  MathSciNet  Google Scholar 

  129. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  130. Zhang, Q., Nian Wu, Y., Zhu, S.-C.: Interpretable convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8827–8836 (2018)

    Google Scholar 

  131. Zhou, Q., Liao, F., Mou, C., Wang, P.: Measuring interpretability for different types of machine learning models. In: Ganji, M., Rashidi, L., Fung, B.C.M., Wang, C. (eds.) PAKDD 2018. LNCS (LNAI), vol. 11154, pp. 295–308. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04503-6_29

    Chapter  Google Scholar 

  132. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Roy. Stat. Soc.: Ser. B (Stat. Methodol.) 67(2), 301–320 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, LMU Munich, Ludwigstr. 33, 80539, Munich, Germany

    Christoph Molnar, Giuseppe Casalicchio & Bernd Bischl

Authors
  1. Christoph Molnar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Casalicchio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernd Bischl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph Molnar .

Editor information

Editors and Affiliations

  1. University of Sydney, Sydney, NSW, Australia

    Irena Koprinska

  2. Monash University, Clayton, VIC, Australia

    Michael Kamp

  3. University of Bari Aldo Moro, Bari, Italy

    Annalisa Appice

  4. University of Bari Aldo Moro, Bari, Italy

    Corrado Loglisci

  5. University of Guelph, Guelph, ON, Canada

    Luiza Antonie

  6. University of Caen Normandy, Caen, France

    Albrecht Zimmermann

  7. University of Pisa, Pisa, Italy

    Riccardo Guidotti

  8. Norwegian University of Science and Technology, Trondheim, Norway

    Özlem Özgöbek

  9. University of Porto, Porto, Portugal

    Rita P. Ribeiro

  10. UPC BarcelonaTech, Barcelona, Spain

    Ricard Gavaldà

  11. University of Porto, Porto, Portugal

    João Gama

  12. Fraunhofer IAIS, St. Augustin, Germany

    Linara Adilova

  13. Royal Holloway University of London, Egham, UK

    Yamuna Krishnamurthy

  14. University of Lisbon, Lisbon, Portugal

    Pedro M. Ferreira

  15. University of Bari Aldo Moro, Bari, Italy

    Donato Malerba

  16. University of Lisbon, Lisbon, Portugal

    Ibéria Medeiros

  17. University of Bari Aldo Moro, Bari, Italy

    Michelangelo Ceci

  18. ICAR-CNR, Rende, Italy

    Giuseppe Manco

  19. University of Naples Federico II, Naples, Italy

    Elio Masciari

  20. University of North Carolina, Charlotte, NC, USA

    Zbigniew W. Ras

  21. Australian National University, Canberra, ACT, Australia

    Peter Christen

  22. Leibniz University Hannover, Hannover, Germany

    Eirini Ntoutsi

  23. Technical University of Dortmund, Dortmund, Germany

    Erich Schubert

  24. University of Southern Denmark, Odense, Denmark

    Arthur Zimek

  25. University of Pisa, Pisa, Italy

    Anna Monreale

  26. Warsaw University of Technology, Warsaw, Poland

    Przemyslaw Biecek

  27. ISTI-CNR, PISA, Italy

    Salvatore Rinzivillo

  28. Berlin Institute of Technology, Berlin, Germany

    Benjamin Kille

  29. Berlin Institute of Technology, Berlin, Germany

    Andreas Lommatzsch

  30. Norwegian University of Science and Technology, Trondheim, Norway

    Jon Atle Gulla

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Molnar, C., Casalicchio, G., Bischl, B. (2020). Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges. In: Koprinska, I., et al. ECML PKDD 2020 Workshops. ECML PKDD 2020. Communications in Computer and Information Science, vol 1323. Springer, Cham. https://doi.org/10.1007/978-3-030-65965-3_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-65965-3_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-65964-6

  • Online ISBN: 978-3-030-65965-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation