Advertisement

Enhanced Transfer Learning with ImageNet Trained Classification Layer

  • Tasfia SherminEmail author
  • Shyh Wei Teng
  • Manzur Murshed
  • Guojun Lu
  • Ferdous Sohel
  • Manoranjan Paul
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11854)

Abstract

Parameter fine tuning is a transfer learning approach whereby learned parameters from pre-trained source network are transferred to the target network followed by fine-tuning. Prior research has shown that this approach is capable of improving task performance. However, the impact of the ImageNet pre-trained classification layer in parameter fine-tuning is mostly unexplored in the literature. In this paper, we propose a fine-tuning approach with the pre-trained classification layer. We employ layer-wise fine-tuning to determine which layers should be frozen for optimal performance. Our empirical analysis demonstrates that the proposed fine-tuning performs better than traditional fine-tuning. This finding indicates that the pre-trained classification layer holds less category-specific or more global information than believed earlier. Thus, we hypothesize that the presence of this layer is crucial for growing network depth to adapt better to a new task. Our study manifests that careful normalization and scaling are essential for creating harmony between the pre-trained and new layers for target domain adaptation. We evaluate the proposed depth augmented networks for fine-tuning on several challenging benchmark datasets and show that they can achieve higher classification accuracy than contemporary transfer learning approaches.

Keywords

CNNs Parameter fine-tuning Depth augmentation 

References

  1. 1.
    Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  2. 2.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  3. 3.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  4. 4.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  5. 5.
    Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)Google Scholar
  6. 6.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)Google Scholar
  7. 7.
    Hariharan, B., Arbeláez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447–456 (2015)Google Scholar
  8. 8.
    Yang, S., Ramanan, D.: Multi-scale recognition with DAG-CNNs. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1215–1223 (2015)Google Scholar
  9. 9.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)Google Scholar
  10. 10.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)
  11. 11.
    Zhang, K., Schölkopf, B., Muandet, K., Wang, Z.: Domain adaptation under target and conditional shift. In: International Conference on Machine Learning, pp. 819–827 (2013)Google Scholar
  12. 12.
    Sigaud, O., Droniou, A.: Towards deep developmental learning. IEEE Trans. Cogn. Dev. Syst. 8, 99–114 (2016)CrossRefGoogle Scholar
  13. 13.
    Tessler, C., Givony, S., Zahavy, T., Mankowitz Daniel J., Mannor, S.: A deep hierarchical approach to lifelong learning in minecraft. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)Google Scholar
  14. 14.
    Pickett, M., Al-Rfou, R., Shao, L., Tar, C.: A growing long-term episodic & semantic memory. arXiv preprint arXiv:1610.06402 (2016)
  15. 15.
    Wang, Y., Ramanan, D., Hebert, M.: Growing a brain: fine-tuning by increasing model capacity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2471–2480 (2017)Google Scholar
  16. 16.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37 (2015)Google Scholar
  17. 17.
    Nilsback, M., Zisserman, A.: Automated flower classification over a large number of classes. In: Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 722–729 (2008)Google Scholar
  18. 18.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset. California Institute of Technology (2011)Google Scholar
  19. 19.
    Aditya K., Nityananda J., Bangpeng Y., Li F.: Novel dataset for fine-grained image categorization. In: First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  20. 20.
    Parkhi, O. M., Vedaldi, A., Zisserman, A., Jawahar, C. V.: Cats and dogs. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  21. 21.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. California Institute of Technology (2007)Google Scholar
  22. 22.
    Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2007 (VOC 2007) Results (2007)Google Scholar
  23. 23.
    Quattoni, A., Torralba, A.: Recognizing indoor scenes. IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–420 (2009)Google Scholar
  24. 24.
    Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A.: SUN database: large-scale scene recognition from abbey to zoo. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3485–3492 (2010)Google Scholar
  25. 25.
    Qian, Q., Jin, R., Zhu, S., Lin, Y.: Fine-grained visual categorization via multi-stage metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3716–3724 (2015)Google Scholar
  26. 26.
    Joulin, A., van der Maaten, L., Jabri, A., Vasilache, N.: Learning visual features from large weakly supervised data. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 67–84. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_5CrossRefGoogle Scholar
  27. 27.
    Azizpour, H., Sharif Razavian, Ali., Sullivan, J., Maki, A., Carlsson, S.: From generic to specific deep representations for visual recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 36–45 (2015)Google Scholar
  28. 28.
    Tamaazousti, Y., Le Borgne, H., Hudelot, C., El Amine Seddik, M., Tamaazousti, M.: Learning more universal representations for transfer-learning. arXiv preprint arXiv:1712.09708 (2017)
  29. 29.
    Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tasfia Shermin
    • 1
    Email author
  • Shyh Wei Teng
    • 1
  • Manzur Murshed
    • 1
  • Guojun Lu
    • 1
  • Ferdous Sohel
    • 2
  • Manoranjan Paul
    • 3
  1. 1.School of Science, Engineering and Information TechnologyFederation UniversityChurchillAustralia
  2. 2.Murdoch UniversityPerthAustralia
  3. 3.Charles Sturt UniversityAlbury-WodongaAustralia

Personalised recommendations