Interpretability of Machine Learning Solutions in Industrial Decision Engineering

  • Inna KolyshkinaEmail author
  • Simeon Simoff
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1127)


The broad application of machine learning (ML) methods and algorithms in diverse range of organisational settings led to the adoption of legislation, like European Union’s General Data Protection Regulation, which require firm capabilities to explain algorithmic decisions. Currently in the ML literature there does not seem to be a consensus on the definition of interpretability of a ML solution. Moreover, there is no agreement about the necessary level of interpretability of such solution and on how this level can be determined, measured and achieved. In this article, we provide such definitions based on research as well as our extensive experience of building ML solutions for various organisations across industries. We present CRISP-ML, a detailed step-by-step methodology, that provides guidance on creating the necessary level of interpretability at each stage of the solution building process and is consistent with the best practices of project management in the ML settings. We illustrate the versatility and effortless applicability of CRISP-ML with examples across a variety of industries and types of ML projects.


Interpretability in machine learning Machine learning methodology Data science methodology Level of interpretability Model interpretability Project management 


  1. 1.
    Big Data and AI executive survey. Technical report, NewVantagePartners LLC (2019)Google Scholar
  2. 2.
    General Data Protection Regulation (GDPR). Official Journal of the European Union L 119/1 (2016).
  3. 3.
    PMBOK® Guide - Sixth Edition. Project Magament Institute (2017)Google Scholar
  4. 4.
    Google AI: Responsible AI Practices - Interpretability. Accessed 5 Aug 2019
  5. 5.
    Ahmed, B., Dannhauser, T., Philip, N.: A Lean Design Thinking Methodology (LDTM) for machine learning and modern data projects. In: Proceedings of 2018 10th Computer Science and Electronic Engineering (CEEC), pp. 11–14. IEEE (2018)Google Scholar
  6. 6.
    Dawson, D., et al.: Artificial intelligence: Australia’s ethics framework. Technical report, Data61 CSIRO, Australia (2019)Google Scholar
  7. 7.
    Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv e-prints arXiv:1702.08608, February 2017
  8. 8.
    Fleming, O., Fountaine, T., Henke, N., Saleh, T.: Ten red flags signaling your analytics program will fail. Technical report, McKinsey & Company (2018)Google Scholar
  9. 9.
    Gilpin, L.H., Testart, C., Fruchter, N., Adebayo, J.: Explaining explanations to society. CoRR abs/1901.06560 (2019).
  10. 10.
    Gleicher, M.: A framework for considering comprehensibility in modeling. Big Data 4(2), 75–88 (2016)CrossRefGoogle Scholar
  11. 11.
    Goodson, M.: Reasons why data projects fail. KDnuggets, November 2016.
  12. 12.
    Grady, N.W., Underwood, M., Roy, A., Chang, W.L.: Big data: challenges, practices and technologies: In: NIST Big Data Public Working Group workshop at IEEE Big Data 2014. Proceedings of IEEE International Conference on Big Data 2014, pp. 11–15 (2014)Google Scholar
  13. 13.
    Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 93:1–93:42 (2018)CrossRefGoogle Scholar
  14. 14.
    Huang, W., McGregor, C., James, A.: A comprehensive framework design for continuous quality improvement within the neonatal intensive care unit: integration of the SPOE, CRISP-DM and PaJMa models. In: Proceedings of IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 289–292 (2014)Google Scholar
  15. 15.
    Larson, D., Chang, V.: A review and future direction of agile, business intelligence, analytics and data science. Int. J. Inf. Manag. 36(5), 700–710 (2016)CrossRefGoogle Scholar
  16. 16.
    Lipton, Z.C.: The mythos of model interpretability. ACM Queue 16(3), 30:31–30:57 (2018)MathSciNetGoogle Scholar
  17. 17.
    Mariscal, G., Marbán, O., Fernández, C.: A survey of data mining and knowledge discovery process models and methodologies. Knowl. Eng. Rev. 25(2), 137–166 (2010)CrossRefGoogle Scholar
  18. 18.
    Molnar, C., Casalicchio, G., Bischl, B.: Quantifying interpretability of arbitrary machine learning models through functional decomposition. arXiv:1904.03867
  19. 19.
    Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Yu, B.: Interpretable machine learning: definitions, methods, and applications. arXiv:1901.04592
  20. 20.
    O’Neil, C.: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishers, New York (2016)zbMATHGoogle Scholar
  21. 21.
    Plotnikova, V.: Towards a data mining methodology for the banking domain. In: Kirikova, M., et al. (ed.) Proceedings of the Doctoral Consortium Papers Presented at the 30th International Conference on Advanced Information Systems Engineering, CAiSE 2018, pp. 46–54 (2018)Google Scholar
  22. 22.
    Ransbotham, S., Kiron, D., Prentice, P.K.: Minding the analytics gap. MIT Sloan Manag. Rev. 56, 1 (2015)Google Scholar
  23. 23.
    Reisman, D., Schultz, J., Crawford, K., Whittaker, M.: Algorithmic impact assessments: a practical framework for public agency accountability. Technical report, AI Now Institute, April 2018Google Scholar
  24. 24.
    Roy Schulte, W., et al.: Predicts 2019: data and analytics strategy. Technical report, Gartner Research, November 2018Google Scholar
  25. 25.
    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019)CrossRefGoogle Scholar
  26. 26.
    Saltz, J.S., Shamshurin, I.: Big data team process methodologies: a literature review and the identification of key factors for a project’s success. In: Proceedings of 2016 IEEE International Conference on Big Data 2016, pp. 2872–2879 (2016)Google Scholar
  27. 27.
    Saltz, J.S., Shamshurin, I., Crowston, K.: Comparing data science project management methodologies via a controlled experiment. In: HICSS (2017)Google Scholar
  28. 28.
    Shearer, C.: The CRISP-DM model: the new blueprint for data mining. J. Data Warehouse. 5, 13–22 (2000)Google Scholar
  29. 29.
    Stieglitz, C.: Beginning at the end - requirements gathering lessons from a flowchart junkie. In: PMI® Global Congress 2012–North America, Vancouver, British Columbia, Canada. Project Management Institute, Newtown Square, PA (2012)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Analytikk ConsultingSydneyAustralia
  2. 2.Western Sydney UniversityPenrithAustralia

Personalised recommendations