Abstract
In recent years, with deep learning achieving a great success, deep transfer learning gradually becomes a new issue. Fine-tuning as a simple transfer learning method can be used to help train deep network and improve the performance of network. In our paper, we use two fine-tuning strategies on deep convolutional neural network and compare their results. There are many influencing factors, such as the depth and width of the network, the amount of data, the similarity of the source and target domain, and so on. Then we keep the network structure and other related factors consistent and use the fine fine-tuning strategy to find the effect of cross-domain factor and similarity of task. Specifically, we use source network and target test data to calculate the similarity. The results of experiments show that when we use fine-tune strategy, using different dataset in source and target domain would affect the target task a lot. Besides the similarity of tasks has direction, and to some extent the similarity would reflect the increment of performance of target task when the source and target task use the same dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) NIPS 2012, pp. 1097–1105 (2012)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE CVPR 2015 (2015)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng., 1345–1359 (2010)
Ge, W., Yu, Y.: Borrowing treasures from the wealthy: deep transfer learning through selective joint fine-tuning. arXiv preprint arXiv:1702.08690 (2017)
Xu, Z., Huang, S., Zhang, Y., Tao, D.: Webly-supervised fine-grained visual categorization via deep domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. (2016)
Long, M., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. arXiv preprint arXiv:1605.06636 (2016)
Ding, Z., Nasrabadi, N.M., Fu, Y.: Task-driven deep transfer learning for image classification. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2414–2418. IEEE (2016)
Azizpour, H., Razavian, A. S., Sullivan, J., Maki, A., Carlsson, S.: Factors of transferability for a generic convNet representation. IEEE Trans. Pattern Anal. Mach. Intell., 1790–1802 (2016)
A visual proof that neural nets can compute any function, neuralnetworksanddeeplearning.com/chap4.html. Accessed 20 May 2017
CS231n Convolutional Neural Networks for Visual Recognition, cs231.github.io/transfer-learning/. Accessed 15 June 2017
Acknowledgments
The work is funded by the National Natural Science Foundation of China (No. 61170155), Shanghai Innovation Action Plan Project (No. 16511101200) and the Open Project Program of the National Laboratory of Pattern Recognition (No. 201600017).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Zhang, W., Fang, Y., Ma, Z. (2017). The Effect of Task Similarity on Deep Transfer Learning. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10635. Springer, Cham. https://doi.org/10.1007/978-3-319-70096-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-70096-0_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70095-3
Online ISBN: 978-3-319-70096-0
eBook Packages: Computer ScienceComputer Science (R0)