Advertisement

Understanding Deep Neural Network by Filter Sensitive Area Generation Network

  • Yang Qian
  • Hong Qiao
  • Jing Xu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11301)

Abstract

Deep convolutional networks have recently gained much attention because of their impressive performance on some visual tasks. However, it is still not clear why they achieve such great success. In this paper, a novel approach called Filter Sensitive Area Generation Network (FSAGN), has been proposed to interpret what the convolutional filters have learnt after training CNNs. Given any trained CNN model, the proposed method aims to figure out which object part each filter represents in a high conv-layer, through appropriate input image mask which filters out unrelated area. In order to obtain such a mask, a mask generation network is designed and the corresponding loss function is defined to evaluate the changes of feature maps before and after mask operation. Experiments on multiple datasets and networks show that FSAGN clarifies the knowledge representations of each filter and how small disturbance on specific object parts affects the performance of CNNs.

Keywords

Convolutional neural network Interpretability Knowledge representations 

Notes

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China (2017YFB1300203), in part by the National Natural Science Foundation of China under Grant 91648205.

References

  1. 1.
    Zhang, N., Donahue, J., Girshick, R., Darrell, T.: Part-based R-CNNs for fine-grained category detection. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 834–849. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_54CrossRefGoogle Scholar
  2. 2.
    Zhang, X., Xiong, H., Zhou, W., Tian, Q.: Picking deep filter responses for fine-grained image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, pp. 1134–1142 (2016)Google Scholar
  3. 3.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: International Conference on Neural Information Processing Systems (NIPS 2015), vol. 39, pp. 91–99. MIT Press (2015)Google Scholar
  4. 4.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Computer Vision and Pattern Recognition (CVPR 2016), pp. 779–788. IEEE Computer Society (2016)Google Scholar
  5. 5.
    Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)CrossRefGoogle Scholar
  6. 6.
    Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: IEEE International Conference on Computer Vision (ICCV 2015), pp. 1520–1528. IEEE Computer Society (2015)Google Scholar
  7. 7.
    Zeiler, Matthew D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_53CrossRefGoogle Scholar
  8. 8.
    Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. In: International Conference on Machine Learning — Deep Learning Workshop, pp. 12 (2015)Google Scholar
  9. 9.
    Kumar, D., Wong, A., Taylor, G.W., Kumar, D., Wong, A., Taylor, G.W.: Explaining the unexplained: A CLass-Enhanced Attentive Response (CLEAR) approach to understanding deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR 2017), pp. 1686–1694. IEEE (2017)Google Scholar
  10. 10.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. (2014)
  11. 11.
    Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151. (2013)
  12. 12.
    Zheng, H., Fu, J., Mei, T., Luo, J.: Learning multi-attention convolutional neural network for fine-grained image recognition. In: IEEE International Conference on Computer Vision, pp. 5219–5227. IEEE Computer Society (2017)Google Scholar
  13. 13.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 248–255. IEEE (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.The State Key Lab of Management and Control for Complex SystemsInstitute of Automation, Chinese Academy of ScienceBeijingChina
  2. 2.University of Chinese Academy of SciencesBeijingChina
  3. 3.CAS Center for Excellence in Brain Science and Intelligence TechnologyShanghaiChina
  4. 4.Cloud Computing CenterChinese Academy of SciencesDongguanChina
  5. 5.Department of Mechanical EngineeringTsinghua UniversityBeijingChina

Personalised recommendations