Layer-Wise Entropy Analysis and Visualization of Neurons Activation

  • Longwei WangEmail author
  • Peijie Chen
  • Chengfei Wang
  • Rui Wang
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 313)


Understanding the inner working mechanism of deep neural networks (DNNs) is essential and important for researchers to design and improve the performance of DNNs. In this work, the entropy analysis is leveraged to study the neurons activation behavior of the fully connected layers of DNNs. The entropy of the activation patterns of each layer can provide an efficient performance metric for the evaluation of the network model accuracy. The study is conducted based on a well trained network model. The activation patterns of shallow and deep layers of the fully connected layers are analyzed by inputting the images of a single class. It is found that for the well trained deep neural networks model, the entropy of the neuron activation pattern is monotonically reduced with the depth of the layers. That is, the neuron activation patterns become more and more stable with the depth of the fully connected layers. The entropy pattern of the fully connected layers can also provide guidelines as to how many fully connected layers are needed to guarantee the accuracy of the model. The study in this work provides a new perspective on the analysis of DNN, which shows some interesting results.


Entropy analysis Visualization Neurons activation 


  1. 1.
    Olah, C., Mordvintsev, A., Schubert, L.: Feature visualization. Distill 2(11), e7 (2017)CrossRefGoogle Scholar
  2. 2.
    Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)
  3. 3.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)CrossRefGoogle Scholar
  4. 4.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR, abs/1512.03385 (2015)Google Scholar
  5. 5.
    Alain, G., Bengio, Y.: Understanding intermediate layers using linear classifier probes (2016)Google Scholar
  6. 6.
    Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. In: Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing (1999)Google Scholar
  7. 7.
    Wang, L., Liang, Q.: Representation learning and nature encoded fusion for heterogeneous sensor networks. IEEE Access 7, 39227–39235 (2019)CrossRefGoogle Scholar
  8. 8.
    Moshkovich, M., Tishby, N.: Mixing complexity and its applications to neural networks (2017). URL
  9. 9.
    Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle. In: Information Theory Workshop (ITW), pp. 1–5. IEEE (2015)Google Scholar
  10. 10.
    Mohamed, S., Rezende, D.V.: Variational information maximisation for intrinsically motivated reinforcement learning. In: NIPS, pp. 2125–2133 (2015)Google Scholar
  11. 11.
    Wang, L., Liang, Q.: Partial interference alignment for heterogeneous cellular networks. IEEE Access 6, 22592–22601 (2018)CrossRefGoogle Scholar
  12. 12.
    Achille, A., Soatto, S.: Information dropout: learning optimal representations through noisy computation (2016). URL
  13. 13.
    Bau, D., Zhou, B., Khosla, A., Oliva, A. and Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6541–6549 (2017)Google Scholar
  14. 14.
    Agostinelli, F., Hoffman, M., Sadowski, P., Baldi, P.: Learning activation functions to improve deep neural networks. arXiv preprint arXiv:1412.6830 (2014)

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2020

Authors and Affiliations

  • Longwei Wang
    • 1
    Email author
  • Peijie Chen
    • 1
  • Chengfei Wang
    • 1
  • Rui Wang
    • 2
  1. 1.Department of Computer Science and Software EngineeringAuburn UniversityAuburnUSA
  2. 2.Department of Information and CommunicationsTongji UniversityShanghaiChina

Personalised recommendations