Abstract
With the deepening of transfer learning research, researchers are no longer satisfied with the classification of knowledge in a single field but hope that the classification of knowledge in multiple fields can be realized, so as to simulate the behavior of human “analogy” and enable the machine to draw inferences”. However, the feature realization of multiple source domains often differs greatly, which brings a challenge to the traditional transfer learning scheme. In this paper, a multi-source deep transfer learning algorithm MDTLFA based on feature alignment is proposed to solve the problem that the data from multiple source domains often has different feature realizations. MDTLFA first reduces the difference in the marginal probability distribution between fields at the sample level by means of the maximum mean deviation MMD. Then, the feature alignment strategy is used at the feature level to further reduce the difference in the marginal probability distribution between the fields and maintain the unique data manifold structure while sharing similar features. On this basis, the conditional probability adaptation CPDA was constructed to reduce the difference in conditional probability distribution between domains and enhance the portability of source domain features. The CPTCNN model was constructed based on a convolutional neural network using CPDA. Finally, the CPTCNN model is trained in the subspace to obtain a classifier set, and the designed strategy is used to select the classifier with a small classification error in the target domain to form MDTLFA. Multiple source domains, marginal probability adaptation at the sample level and feature level, and the CPTCNN model constructed based on the minimization of conditional probability differences effectively improve the performance of data features in multiple domains, thus improving the classification effect. The experimental results on several real data sets show that the MDTLFA algorithm is effective and has some advantages compared with the advanced benchmark algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bang SH et al (2019) A survey on knowledge transfer for manufacturing data analytics. Comput Ind 104:116–130
Blitzer J, Dredze M, Pereira F (2007) Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th annual meeting of the association of computational linguistics. pp 440–447
Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27
Christodoulidis S, Anthimopoulos M, Ebner L et al (2016) Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J Biomed Health Inform 21(1):76–84
Courty N, Flamary R, Habrard A et al (2017) Joint distribution optimal transportation for domain adaptation. Adv Neural Inf Process Syst 30:3730–3739
Ding Z, Shao M, Fu Y (2018) Incomplete multisource transfer learning. IEEE Trans Neural Netw Learn Syst 29(2):310–323
Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. pp 109–117
Fang M, Guo Y, Zhang X et al (2015) Multi-source transfer learning based on label shared subspace. Pattern Recogn Lett 51:101–106
Gao P, Li J et al (2021) Multi-source fast transfer learning algorithm based on support vector machine. Appl Intell 59:8451–8465
Gao P, Li P et al (2022a) Multisource mobile transfer learning algorithm based on dynamic model compression. Secur Commun Netw 2022:1–12
Gao P, Li J et al (2022b) Multi-source deep transfer learning based on balanced distribution adaptation. Comput Intell Neurosci 2022:1–12
Ge X, Yu S, Lin M et al (2017) A selective transfer learning method for concept drift adaptation. In: International symposium on neural networks. Springer, Cham, pp 353–361
Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset[EB/OL]. https://authors.library.caltech.edu/7694/?ref=https://githubhelp.com. Accessed 30 June 2021
Ham J, Lee D, Saul L (2005) Semisupervised alignment of manifolds. In: Proceedings of the 10th international workshop on artificial intelligence and statistics. pp 120–127
Huang ML, Chuang TC, Liao YC (2022) Application of transfer learning and image augmentation technology for tomato pest identification. Sustain Comput Inform Syst 33:100646
Kim Y-M, Shin S-J, Cho H-W (2022) Predictive modeling for machining power based on multi-source transfer learning in metal cutting. Int J Precis Eng Manuf Green Technol 9:107–125
Li X, Mao W, Jiang W (2016) Extreme learning machine based transfer learning for data classification. Neurocomputing 174:203–221
Long M, Wang J, Ding G et al (2014) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089
Long M, Wang J, Sun J et al (2015) Domain invariant transfer kernel learning. IEEE Trans Knowl Data Eng 27(6):1519–1532
Luo L, Wang X, Hu S et al (2019) A unified framework for interactive image segmentation via Fisher rules. Vis Comput 35(12):1869–1882
Ma Y, Xu J et al (2017) A visual analytical approach for transfer learning in classification. Inf Sci 390:54–69
Mozafari AS, Jamzad M (2016) A SVM-based model-transferring method for heterogeneous domain adaptation. Pattern Recogn 56:142–158
Netzer Y, Wang T, Coates A et al (2011) Reading digits in natural images with unsupervised feature learning. In: Nips workshop on deep learning and unsupervised feature learning. p 5
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Pan SJ, Tsang IW, Kwok JT et al (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Peng X, Bai Q, Xia X et al (2019) Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 1406–1415
Shen J, Qu Y, Zhang W et al (2018) Wasserstein distance guided representation learning for domain adaptation. In: Thirty-second AAAI conference on artificial intelligence. pp 4058–4065
Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92
Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. Proc AAAI Conf Artif Intell 30(1):8
Venkateswara H, Eusebio J, Chakraborty S et al (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 5018–5027
Wu Q, Zhou X, Yan Y et al (2017) Online transfer learning by leveraging multiple source domains. Knowl Inf Syst 52(3):687–707
Xu R, Chen Z, Zuo W et al (2018) Deep cocktail network: multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3964–3973
Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive SVMs. In: Proceedings of the 15th international conference on multimedia 2007, Augsburg, Germany, September 24–29, 2007. ACM
Yang S, Kong X, Wang Q et al (2021) A multi-source ensemble domain adaptation method for rotary machine fault diagnosis. Measurement 186:110213
Yao Y, Doretto G (2010) Boosting for transfer learning with multiple sources. In: 2010 IEEE Computer Society Conference on computer vision and pattern recognition. IEEE
Zadrozny B (2004) Learning and evaluating classifiers under sample selection bias. In: Proceedings of the twenty-first international conference on machine learning. p 114
Zellinger W et al (2020) Multi-source transfer learning of time series in cyclical manufacturing. J Intell Manuf 31:777–787
Zhao H, Zhang S, Wu G et al (2018) Adversarial multiple source domain adaptation. Adv Neural Inf Process Syst 31:8559–8570
Zhao H, Des Combes RT, Zhang K et al (2019) On learning invariant representation for domain adaptation. In: International conference on machine learning. pp 7523–7532
Zhu Y, Zhuang F, Wang D (2019) Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. Proc AAAI Conf Artif Intell 33(01):5989–5996
Acknowledgements
This work has been supported by China’s national key research and development plan. (2016YFB0801004), Science and Technology Major Special Project of Heilongjiang (CN) (2020ZX14A02), Natural Science Foundation of Heilongjiang Province (C2016053, LH2020H093), Scientific research project of traditional Chinese medicine in Heilongjiang Province (general survey of traditional Chinese medicine resources 2018hljzyzypc-21), Project of “Support plan for young backbone teachers” of Heilongjiang University of traditional Chinese medicine (15041190011).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ding, C., Gao, P., Li, J. et al. Multi-source deep transfer learning algorithm based on feature alignment. Artif Intell Rev 56 (Suppl 1), 769–791 (2023). https://doi.org/10.1007/s10462-023-10545-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-023-10545-w