Skip to main content
View expanded cover

International Workshop on Health Intelligence

W3PHAI 2019: Precision Health and Medicine pp 97–110Cite as

Explaining Multi-label Black-Box Classifiers for Health Applications

Part of the Studies in Computational Intelligence book series (SCI,volume 843)

Abstract

Today the state-of-the-art performance in classification is achieved by the so-called “black boxes”, i.e. decision-making systems whose internal logic is obscure. Such models could revolutionize the health-care system, however their deployment in real-world diagnosis decision support systems is subject to several risks and limitations due to the lack of transparency. The typical classification problem in health-care requires a multi-label approach since the possible labels are not mutually exclusive, e.g. diagnoses. We propose MARLENA, a model-agnostic method which explains multi-label black box decisions. MARLENA explains an individual decision in three steps. First, it generates a synthetic neighborhood around the instance to be explained using a strategy suitable for multi-label decisions. It then learns a decision tree on such neighborhood and finally derives from it a decision rule that explains the black box decision. Our experiments show that MARLENA performs well in terms of mimicking the black box behavior while gaining at the same time a notable amount of interpretability through compact decision rules, i.e. rules with limited length.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-24409-5_9
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-24409-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3

Notes

  1. 1.

    http://reference.wolfram.com/language/ref/NormalizedSquaredEuclideanDistance.html.

  2. 2.

    For both neighborhood generation approaches mixed and union, the size of the synthetic neighborhood is 1000, and the size of the core real neighborhood \(X^*\) is \(k = 0.5 |\hat{X}|^{1/2}\).

  3. 3.

    Source code, datasets, and the scripts for reproducing experiments are publicly available at https://github.com/riccotti/ExplainMultilabelClassifiers.

  4. 4.

    https://tinyurl.com/y9maxnxr, https://tinyurl.com/yaz2lyrc.

  5. 5.

    We replace the missing values with the mean for continuous variables and with the mode for categorical ones. We remove the features with more than 40% of missing values.

  6. 6.

    Implementations are those of scikit-learn library.

  7. 7.

    Details available at https://github.com/riccotti/ExplainMultilabelClassifiers.

  8. 8.

    The performance reported consider only instances for which an explanation is returned. Indeed, for some instances of the medical dataset using the RF black box an explanation is not returned. We leave the investigation of this specific case fur future studies.

References

  1. Abe, S.: Fuzzy support vector machines for multilabel classification. PR 48(6), 2110 (2015)

    CrossRef  Google Scholar 

  2. Bai, T., et al.: Interpretable representation learning for healthcare via capturing disease progression through time. In: KDD, pp. 43–51. ACM (2018)

    Google Scholar 

  3. Blockeel, H., Schietgat, L., Struyf, J., Clare, A., Dzeroski, S.: Hierarchical multilabel classification trees for gene function prediction. In: MLSB, pp. 9–14 (2006)

    Google Scholar 

  4. Che, Z., Kale, D., Li, W., Bahadori, M.T., Liu, Y.: Deep computational phenotyping. In: KDD, pp. 507–516. ACM (2015)

    Google Scholar 

  5. Che, Z., Purushotham, S., Khemani, R., Liu, Y.: Interpretable deep models for ICU outcome prediction. In: AMIA Annual Symposium Proceedings, vol. 2016, p. 371. American Medical Informatics Association (2017)

    Google Scholar 

  6. Choi, E., et al.: Doctor AI: predicting clinical events via recurrent neural networks. In: Machine Learning for Healthcare Conference, pp. 301–318 (2016)

    Google Scholar 

  7. Chui, M.: Artificial intelligence the next digital frontier? McKinsey and CGI, p. 47 (2017)

    Google Scholar 

  8. Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Advances in Neural Information Processing Systems, pp. 681–687 (2002)

    Google Scholar 

  9. Feng, Y., et al.: Patient outcome prediction via convolutional neural networks based on multi-granularity medical concept embedding. In: BIBM, pp. 770–777. IEEE (2017)

    Google Scholar 

  10. Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., Giannotti, F.: Local rule-based explanations of black box decision systems. CoRR (2018). arXiv:abs/1805.10820

  11. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Pedreschi, D., Giannotti, F.: A survey of methods for explaining black box models. ACM CSUR 51(5), 93:1–93:42 (2018)

    CrossRef  Google Scholar 

  12. Guidotti, R., Soldani, J., Neri, D., Brogi, A., Pedreschi, D.: Helping your Docker images to spread based on explainable models. In: ECML-PKDD. Springer, Berlin (2018)

    Google Scholar 

  13. Lasko, T.A., et al.: Computational phenotype discovery using unsupervised feature learning over noisy, sparse, and irregular clinical data. PloS One 8(6), e66341 (2013)

    CrossRef  Google Scholar 

  14. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    CrossRef  Google Scholar 

  15. Malgieri, G., Comandé, G.: Why a right to legibility of automated decision-making exists in the general data protection regulation. Int. Data Priv. Law 7(4), 243–265 (2017)

    CrossRef  Google Scholar 

  16. Miotto, R., et al.: Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci. Rep. 6, 26094 (2016)

    CrossRef  Google Scholar 

  17. Pestian, J.P., et al.: A shared task involving multi-label classification of clinical free text. In: BioNLP, pp. 97–104. Association for Computational Linguistics (2007)

    Google Scholar 

  18. Rajkomar, A., et al.: Scalable and accurate deep learning with EHR. DM 1(1), 18 (2018)

    Google Scholar 

  19. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. In: KDD, pp. 1135–1144. ACM (2016)

    Google Scholar 

  20. Sapozhnikova, E.P.: Art-based neural networks for multi-label classification. In: International Symposium on Intelligent Data Analysis, pp. 167–177. Springer, Berlin (2009)

    CrossRef  Google Scholar 

  21. Shickel, B., et al.: Deep EHR: a survey of recent advances in deep learning techniques for EHR analysis. J. Biomed. Health Inform. 22(5), 1589–1604 (2018)

    CrossRef  Google Scholar 

  22. Tan, P.-N. et al.: Introduction to data mining. Pearson Education India (2007)

    Google Scholar 

  23. Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehous. Min. (IJDWM) 3(3), 1–13 (2007)

    CrossRef  Google Scholar 

  24. Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Mach. Learn. 73(2), 185 (2008)

    CrossRef  Google Scholar 

  25. Wachter, S., et al.: Why a right to explanation of automated decision-making does not exist in the general data protection regulation. Int. Data Priv. Law 7(2), 76–99 (2017)

    CrossRef  Google Scholar 

  26. Yadav, P., Steinbach, M., Kumar, V., Simon, G.: Mining electronic health records (EHRS): a survey. ACM Comput. Surv. (CSUR) 50(6), 85 (2018)

    CrossRef  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the European H2020 Program under the funding scheme “INFRAIA-1-2014-2015: Research Infrastructures” g.a. 654024 “SoBigData”, http://www.sobigdata.eu.

Author information

Authors and Affiliations

  1. Scuola Normale Superiore, Pisa, Italy

    Cecilia Panigutti

  2. ISTI-CNR, Pisa, Italy

    Riccardo Guidotti

  3. University of Pisa, Pisa, Italy

    Riccardo Guidotti, Anna Monreale & Dino Pedreschi

Authors
  1. Cecilia Panigutti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riccardo Guidotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna Monreale
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dino Pedreschi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cecilia Panigutti .

Editor information

Editors and Affiliations

  1. Department of Pediatrics, The University of Tennessee Health Science Center – Oak-Ridge National Lab (UTHSC-ORNL) Center for Biomedical Informatics, Memphis, TN, USA

    Prof. Arash Shaban-Nejad

  2. School of Nursing, University of Minnesota, Minneapolis, MN, USA

    Prof. Martin Michalowski

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Panigutti, C., Guidotti, R., Monreale, A., Pedreschi, D. (2020). Explaining Multi-label Black-Box Classifiers for Health Applications. In: Shaban-Nejad, A., Michalowski, M. (eds) Precision Health and Medicine. W3PHAI 2019. Studies in Computational Intelligence, vol 843. Springer, Cham. https://doi.org/10.1007/978-3-030-24409-5_9

Download citation