Skip to main content

Advertisement

Log in

Interpretability of artificial neural network models in artificial intelligence versus neuroscience

  • Comment
  • Published:

From Nature Machine Intelligence

View current issue Submit your manuscript

The notion of ‘interpretability’ of artificial neural networks (ANNs) is of growing importance in neuroscience and artificial intelligence (AI). But interpretability means different things to neuroscientists as opposed to AI researchers. In this article, we discuss the potential synergies and tensions between these two communities in interpreting ANNs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1: Interpretability of models for AI and neuroscience.

References

  1. Yamins, D. L. K. et al. Proc. Natl Acad. Sci. USA 111, 8619–8624 (2014).

    Article  Google Scholar 

  2. Schrimpf, M. et al. Proc. Natl Acad. Sci. USA 118, e2105646118 (2021).

    Article  Google Scholar 

  3. Pospisil, D. A., Pasupathy, A. & Bair, W. Elife 7, e38242 (2018).

    Article  Google Scholar 

  4. Bao, P., She, L., McGill, M. & Tsao, D. Y. Nature 583, 103–108 (2020).

    Article  Google Scholar 

  5. Schrimpf, M. et al. Preprint at bioRxiv http://biorxiv.org/lookup/doi/10.1101/407007 (2018).

  6. Kar, K., Kubilius, J., Schmidt, K., Issa, E. B. & DiCarlo, J. J. Nat. Neurosci. 22, 974–983 (2019).

    Article  Google Scholar 

  7. Kar, K. & DiCarlo, J. J. Neuron 109, 164–176 (2021).

    Article  Google Scholar 

  8. European Parliament. Directorate General for Parliamentary Research Services. A governance framework for algorithmic accountability and transparency. (Publications Office, 2019).

  9. Mordvintsev, A., Olah, C., & Tyka, M. Inceptionism: Going deeper into neural networks (2015); https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html

  10. Majaj, N. J., Hong, H., Solomon, E. A. & DiCarlo, J. J. J. Neurosci. 35, 13402–13418 (2015).

    Article  Google Scholar 

  11. Willeke, K. F. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2206.08666 (2022).

  12. Conwell, C. et al. SVRHM 2021 Workshop (NeurIPS, 2021).

  13. Holzinger, A. in 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) 55–66 (IEEE, 2018).

  14. Hooker, S., Erhan, D., Kindermans, P.-J. & Kim, B. Advances in Neural Information Processing Systems 32 (2019).

  15. Gosselin, F. & Schyns, P. G. Vision Res. 41, 2261–2271 (2001).

    Article  Google Scholar 

  16. Murray, R. F. J. Vis. 11, 2 (2011).

    Article  Google Scholar 

  17. Bashivan, P., Kar, K. & DiCarlo, J. J. Science 364, eaav9436 (2019).

    Article  Google Scholar 

  18. Ponce, C. R. et al. Cell 177, 999–1009 (2019).

    Article  Google Scholar 

  19. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. In International conference on computer vision 618–626 (IEEE, 2017).

  20. Geirhos, R. et al. Advances in Neural Information Processing Systems 34, 23885–23899 (2021).

  21. Zipser, D. & Andersen, R. A. Nature 331, 679–684 (1988).

    Article  Google Scholar 

  22. Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Nature 503, 78–84 (2013).

    Article  Google Scholar 

  23. Olshausen, B. A. & Field, D. J. Nature 381, 607–609 (1996).

    Article  Google Scholar 

  24. Olah, C., Mordvintsev, A., Schubert, L. Feature Visualization (Distill, 2017); https://distill.pub/2017/feature-visualization

Download references

Acknowledgements

The authors would like to thank C. Shain for helpful comments and discussions. E.F. was supported by NIH awards R01-DC016607, R01-DC016950 and U01-NS121471, and by research funds from the McGovern Institute for Brain Research, the Brain and Cognitive Sciences Department and the Simons Center for the Social Brain. K.K. was supported by the Canada Research Chair Program. This research was undertaken thanks in part to funding from the Canada First Research Excellence Fund. K.K. was supported by an unrestricted research fund from Google LLC.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kohitij Kar.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kar, K., Kornblith, S. & Fedorenko, E. Interpretability of artificial neural network models in artificial intelligence versus neuroscience. Nat Mach Intell 4, 1065–1067 (2022). https://doi.org/10.1038/s42256-022-00592-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-022-00592-3

  • Springer Nature Limited

This article is cited by

Navigation