Skip to main content

Interpretable Deep Learning in Drug Discovery

Part of the Lecture Notes in Computer Science book series (LNAI,volume 11700)

Abstract

Without any means of interpretation, neural networks that predict molecular properties and bioactivities are merely black boxes. We will unravel these black boxes and will demonstrate approaches to understand the learned representations which are hidden inside these models. We show how single neurons can be interpreted as classifiers which determine the presence or absence of pharmacophore- or toxicophore-like structures, thereby generating new insights and relevant knowledge for chemistry, pharmacology and biochemistry. We further discuss how these novel pharmacophores/toxicophores can be determined from the network by identifying the most relevant components of a compound for the prediction of the network. Additionally, we propose a method which can be used to extract new pharmacophores from a model and will show that these extracted structures are consistent with literature findings. We envision that having access to such interpretable knowledge is a crucial aid in the development and design of new pharmaceutically active molecules, and helps to investigate and understand failures and successes of current methods.

Keywords

  • Deep learning
  • Neural networks
  • Drug development
  • Target prediction

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-28954-6_18
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-28954-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Fig. 18.1.
Fig. 18.2.
Fig. 18.3.
Fig. 18.4.
Fig. 18.5.
Fig. 18.6.
Fig. 18.7.

References

  1. Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks (2018)

    Google Scholar 

  2. Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., Müller, K.R.: How to explain individual classification decisions. J. Mach. Learn. Res. 11, 1803–1831 (2010). http://dl.acm.org/citation.cfm?id=1756006.1859912

    MathSciNet  MATH  Google Scholar 

  3. Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6541–6549 (2017)

    Google Scholar 

  4. Bengio, Y.: Deep learning of representations: looking forward. In: Dediu, A.-H., Martín-Vide, C., Mitkov, R., Truthe, B. (eds.) SLSP 2013. LNCS (LNAI), vol. 7978, pp. 1–37. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39593-2_1

    CrossRef  Google Scholar 

  5. Cherkasov, A., et al.: QSAR modeling: where have you been? Where are you going to? J. Med. Chem. 57(12), 4977–5010 (2014)

    CrossRef  Google Scholar 

  6. Chollet, F.: Keras (2015). https://keras.io

  7. Duvenaud, D.K., et al.: Convolutional networks on graphs for learning molecular fingerprints. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2224–2232. Curran Associates, Inc. (2015)

    Google Scholar 

  8. Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, PMLR, International Convention Centre, Sydney, Australia, vol. 70, pp. 1263–1272, 06–11 August 2017

    Google Scholar 

  9. Hansen, K., Baehrens, D., Schroeter, T., Rupp, M., Müller, K.R.: Visual interpretation of kernel-based prediction models. Mol. Inf. 30(9), 817–826 (2011)

    CrossRef  Google Scholar 

  10. Hansen, K., et al.: Benchmark data set for in silico prediction of ames mutagenicity. J. Chem. Inf. Modeling 49(9), 2077–2081 (2009)

    CrossRef  Google Scholar 

  11. Huang, R., et al.: Profiling of the Tox21 10K compound library for agonists and antagonists of the estrogen receptor alpha signaling pathway. Sci. Rep. 4 (2014)

    Google Scholar 

  12. Kazius, J., McGuire, R., Bursi, R.: Derivation and validation of toxicophores for mutagenicity prediction. J. Med. Chem. 48(1), 312–320 (2005)

    CrossRef  Google Scholar 

  13. Kearnes, S., McCloskey, K., Berndl, M., Pande, V., Riley, P.: Molecular graph convolutions: moving beyond fingerprints. J. Comput. Aided Mol. Des. 30(8), 595–608 (2016)

    CrossRef  Google Scholar 

  14. Klambauer, G., Unterthiner, T., Mayr, A., Hochreiter, S.: Self-normalizing neural networks. In: Advances in Neural Information Processing Systems 30 (NIPS) (2017)

    Google Scholar 

  15. Lavecchia, A.: Machine-learning approaches in drug discovery: methods and applications. Drug Discovery Today 20(3), 318–331 (2015)

    CrossRef  Google Scholar 

  16. Lin, S.: Pharmacophore perception, development and use in drug design. Edited by Osman F. Güner. Molecules 5(7), 987–989 (2000)

    CrossRef  Google Scholar 

  17. Lionta, E., Spyrou, G., Vassilatis, D.K., Cournia, Z.: Structure-based virtual screening for drug discovery: principles, applications and recent advances. Curr. Top. Med. Chem. 14(16), 1923–1938 (2014)

    CrossRef  Google Scholar 

  18. Lounkine, E., et al.: Large-scale prediction and testing of drug activity on side-effect targets. Nature 486(7403), 361–367 (2012). https://doi.org/10.1038/nature11159

    CrossRef  Google Scholar 

  19. Ma, J., Sheridan, R.P., Liaw, A., Dahl, G.E., Svetnik, V.: Deep neural nets as a method for quantitative structure-activity relationships. J. Chem. Inf. Model. 55(2), 263–274 (2015)

    CrossRef  Google Scholar 

  20. Mayr, A., Klambauer, G., Unterthiner, T., Hochreiter, S.: DeepTox: toxicity prediction using deep learning. Frontiers Environ. Sci. 3, 80 (2016)

    CrossRef  Google Scholar 

  21. Mayr, A., et al.: Large-scale comparison of machine learning methods for drug target prediction on ChEMBL. Chem. Sci. 9(24), 5441–5451 (2018)

    CrossRef  Google Scholar 

  22. Olivecrona, M., Blaschke, T., Engkvist, O., Chen, H.: Molecular de-novo design through deep reinforcement learning. J. Cheminform. 9(1), 48 (2017)

    CrossRef  Google Scholar 

  23. Plošnik, A., Vračko, M., Dolenc, M.S.: Mutagenic and carcinogenic structural alerts and their mechanisms of action. Arch. Ind. Hyg. Toxicol. 67(3), 169–182 (2016)

    Google Scholar 

  24. Preuer, K., Lewis, R.P.I., Hochreiter, S., Bender, A., Bulusu, K.C., Klambauer, G.: DeepSynergy: predicting anti-cancer drug synergy with deep learning. Bioinformatics 34(9), 1538–1546 (2017)

    CrossRef  Google Scholar 

  25. Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S., Klambauer, G.: Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. J. Chem. Inf. Model. 58(9), 1736–1741 (2018)

    CrossRef  Google Scholar 

  26. Rogers, D., Hahn, M.: Extended-connectivity fingerprints. J. Chem. Inf. Model. 50(5), 742–754 (2010)

    CrossRef  Google Scholar 

  27. Schütt, K.T., Arbabzadah, F., Chmiela, S., Müller, K.R., Tkatchenko, A.: Quantum-chemical insights from deep tensor neural networks. Nat. Commun. 8, 13890 (2017)

    CrossRef  Google Scholar 

  28. Segler, M.H., Kogej, T., Tyrchan, C., Waller, M.P.: Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Sci. (2017)

    Google Scholar 

  29. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML) (2017)

    Google Scholar 

  30. Sushko, I., Salmina, E., Potemkin, V.A., Poda, G., Tetko, I.V.: ToxAlerts: a web server of structural alerts for toxic chemicals and compounds with potential adverse reactions. J. Chem. Inf. Model. 52(8), 2310–2316 (2012)

    CrossRef  Google Scholar 

  31. Unterthiner, T., et al.: Multi-task deep networks for drug target prediction. In: Workshop on Transfer and Multi-task Learning of NIPS 2014, vol. 2014, pp. 1–4 (2014)

    Google Scholar 

  32. Unterthiner, T., et al.: Deep learning as an opportunity in virtual screening. In: Deep Learning and Representation Learning Workshop (NIPS 2014) (2014)

    Google Scholar 

  33. Unterthiner, T., Nessler, B., Klambauer, G., Heusel, M., Ramsauer, H., Hochreiter, S.: Coulomb GANs: provably optimal Nash equilibria viapotential fields. In: International Conference of Learning Representations (ICLR) (2018)

    Google Scholar 

  34. Yang, H., Li, J., Wu, Z., Li, W., Liu, G., Tang, Y.: Evaluation of different methods for identification of structural alerts using chemical ames mutagenicity data set as a benchmark. Chem. Res. Toxicol. 30(6), 1355–1364 (2017)

    CrossRef  Google Scholar 

  35. Yang, X., Zhang, J., Yoshizoe, K., Terayama, K., Tsuda, K.: ChemTS: an efficient python library for de novo molecular generation. Sci. Technol. Adv. Mater. 18(1), 972–976 (2017)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas Unterthiner .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Preuer, K., Klambauer, G., Rippmann, F., Hochreiter, S., Unterthiner, T. (2019). Interpretable Deep Learning in Drug Discovery. In: Samek, W., Montavon, G., Vedaldi, A., Hansen, L., Müller, KR. (eds) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Lecture Notes in Computer Science(), vol 11700. Springer, Cham. https://doi.org/10.1007/978-3-030-28954-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-28954-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-28953-9

  • Online ISBN: 978-3-030-28954-6

  • eBook Packages: Computer ScienceComputer Science (R0)