Critical Challenges for the Visual Representation of Deep Neural Networks

  • Kieran BrowneEmail author
  • Ben Swift
  • Henry Gardner
Part of the Human–Computer Interaction Series book series (HCIS)


Artificial neural networks have proved successful in a broad range of applications over the last decade. However, there remain significant concerns about their interpretability. Visual representation is one way researchers are attempting to make sense of these models and their behaviour. The representation of neural networks raises questions which cross disciplinary boundaries. This chapter draws on a growing collection of interdisciplinary scholarship regarding neural networks. We present six case studies in the visual representation of neural networks and examine the particular representational challenges posed by these algorithms. Finally we summarise the ideas raised in the case studies as a set of takeaways for researchers engaging in this area.



We are grateful for the helpful advice of Mitchell Whitelaw throughout the development of this chapter.


  1. 1.
    Andrews, F.: CPPNX. (2017). Accessed 14 Dec 2017
  2. 2.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 (2014)
  3. 3.
    Barocas, S., Selbst, A.: Big data’s disparate impact. Calif. Law Rev. 104(671), 671–732 (2016).
  4. 4.
    Bastian, M., Heymann, S., Jacomy, M.: Others: Gephi: an open source software for exploring and manipulating networks. Icwsm 8, 361–362 (2009)Google Scholar
  5. 5.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)CrossRefGoogle Scholar
  6. 6.
    Benitez, J., Castro, J., Requena, I.: Are artificial neural networks black boxes? IEEE Trans. Neural Netw. 8(5), 1156–1164 (1997)CrossRefGoogle Scholar
  7. 7.
    Breiman, L.: Statistical modeling: the two cultures. Stat. Sci. 16(3), 199–231 (2001)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Browne, K.: Interpretable long-range LSTM cells visualisation redrawn from Karpathy Johnson and Fei-Fei 2015. (2017). Accessed 11 Dec 2017
  9. 9.
    Browne, K.: Neural interpretation diagram redrawn from özesmi and özesmi 2005. (2017). Accessed 11 Dec 2017
  10. 10.
    Browne, K.: Neural interpretation diagram redrawn from TZENG and MA 2005. (2017). Accessed 18 Dec 2017
  11. 11.
    Browne, K.: Script for fooling images as in Nguyen Yosinski Clune (2015). (2017). Accessed 13 Dec 2017
  12. 12.
    Burgess, M.: Gallery: ‘Brain scans’ map what happens during inside machine learning. Accessed 13 Aug 2017
  13. 13.
    Burrell, J.: How the machine thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 3(1) (2016)CrossRefGoogle Scholar
  14. 14.
    Craven, M.W., Shavlik, J.W.: Visualizing learning and computation in artificial neural networks. Int. J. Artif. Intell. Tools 1(3), 399–425 (1992)CrossRefGoogle Scholar
  15. 15.
    Cresci, E.: FaceApp apologises for ‘racist’ filter that lightens users’ skintone. (2017). Accessed 30 Aug 2017
  16. 16.
    Duch, W.: Coloring black boxes: visualization of neural network decisions. In: Proceedings of the International Joint Conference on Neural Networks, vol. 3, pp. 1735–1740. IEEE (2003)Google Scholar
  17. 17.
    Elgammal, A., Liu, B., Elhoseiny, M., Mazzone, M.: CAN: creative adversarial networks, generating “Art” by learning about styles and deviating from style norms. arXiv:1706.07068 (2017)
  18. 18.
    FaceApp: Faceapp - free neural face transformation filters. (2017). Accessed 21 Dec 2017
  19. 19.
    Fyles, M.: Inside an AI ’brain’ - What does machine learning look like? (2017). Accessed 13 Aug 2017
  20. 20.
    Fyles, M.: Neural network structure, MSR ResNet-50 - large directed graph visualization [OC] : dataisbeautiful. Accessed 13 Aug 2017
  21. 21.
    Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)CrossRefGoogle Scholar
  22. 22.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv:1611.07004 (2016)
  23. 23.
    Karpathy, A., Johnson, J., Li, F.: Visualizing and understanding recurrent networks. CoRR arXiv:1506.02078 (2015)
  24. 24.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25(NIPS2012), 1–9 (2012)Google Scholar
  25. 25.
    Längkvist, M., Karlsson, L., Loutfi, A.: A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit. Lett. 42, 11–24 (2014)CrossRefGoogle Scholar
  26. 26.
    Le, Q.V.: Building high-level features using large scale unsupervised learning. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8595–8598. IEEE (2013)Google Scholar
  27. 27.
    Manovich, L.: The anti-sublime ideal in data art. (2002). Accessed 21 Dec 2017
  28. 28.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv:1312.5602 (2013)
  29. 29.
    Mordvintsev, A., Olah, C., Tyka, M.: Inceptionism: going deeper into neural networks. Accessed 12 Aug 2017
  30. 30.
    Mordvintsev, A., Tyka, M., Olah, C.: Deep dreams (with Caffe). (2017). Accessed 09 May 2017
  31. 31.
    Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)Google Scholar
  32. 32.
    Olden, J.D., Jackson, D.A.: Illuminating the "black box": a randomization approach for understanding variable contributions in artificial neural networks. Ecol. Model. 154(1–2), 135–150 (2002)CrossRefGoogle Scholar
  33. 33.
    Özesmi, S.L., Özesmi, U.: An artificial neural network approach to spatial habitat modelling with interspecific interaction. Ecol. Model. 116(1), 15–31 (1999)CrossRefGoogle Scholar
  34. 34.
    Rosenblatt, F.: Principles of neurodynamics: perceptrons and the theory of brain mechanics. Spartan Book (1962)Google Scholar
  35. 35.
    Sainath, T.N., Vinyals, O., Senior, A., Sak, H.: Convolutional, long short-term memory, fully connected deep neural networks. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4580–4584. IEEE (2015)Google Scholar
  36. 36.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. CoRR arXiv:1312.6034 (2013)
  37. 37.
    Stanley, K.O.: Compositional pattern producing networks: a novel abstraction of development. Genet. Program. Evolvable Mach. 8(2), 131–162 (2007)CrossRefGoogle Scholar
  38. 38.
    Streeter, M., Ward, M., Alvarez, S.A.: NVIS: an interactive visualization tool for neural networks. In: Proceedings of Visual Data Exploration and Analysis Conference (2001)Google Scholar
  39. 39.
    Sussillo, D., Barak, O.: Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput. 25(3), 626–49 (2013)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Tensorflow: Tensorboard: Graph visualization. (2017). Accessed 21 Dec 2017
  41. 41.
    Tunç, B.: Semantics of object representation in machine learning. Pattern Recognit. Lett. 64, 30–36 (2015). Scholar
  42. 42.
    Tzeng, F.Y., Ma, K.L.: Opening the black box-data driven visualization of neural networks. Proceedings of IEEE Visualization 2005, 383–390 (2005)Google Scholar
  43. 43.
    Zahavy, T., Ben-Zrihem, N., Mannor, S.: Graying the black box: understanding DQNS. In: International Conference on Machine Learning, pp. 1899–1908 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Australian National UniversityCanberraAustralia

Personalised recommendations