Skip to main content
SpringerLink
Log in

Introduction

  • Chapter
  • First Online:
Introduction to Transfer Learning

  • 1862 Accesses

Abstract

In this chapter, we introduce the background of transfer learning to give an overview of this area. This chapter can be thought of as a broad introduction to readers who have never experienced transfer learning. Thus, this chapter is self-contained. Experienced readers can skip it with no harm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info
Hardcover Book
USD 79.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    These free images are from https://pixabay.com/.

  2. 2.

    https://icml.cc/.

  3. 3.

    https://neurips.cc/.

  4. 4.

    https://iclr.cc/.

  5. 5.

    http://plato.acadiau.ca/courses/comp/dsilver/NIPS95_LTL/transfer.workshop.1995.html.

  6. 6.

    http://logic.stanford.edu/tl/TransferLearningPIP.pdf.

  7. 7.

    http://orca.st.usm.edu/~banerjee/icmlws06/.

  8. 8.

    https://eecs.wsu.edu/~taylorm/AAAI08TL/index.htm.

  9. 9.

    http://clopinet.com/isabelle/Projects/ICML2011/home.html.

  10. 10.

    http://www.causality.inf.ethz.ch/unsupervised-learning.php.

  11. 11.

    https://sites.google.com/site/learningacross/.

  12. 12.

    http://adas.cvc.uab.es/task-cv2017/, https://sites.google.com/view/task-cv2019/home.

  13. 13.

    https://lld-workshop.github.io/.

  14. 14.

    https://www.kdnuggets.com/news/2007/n15/7i.html.

  15. 15.

    https://zhuanlan.zhihu.com/p/40631601.

  16. 16.

    https://www.msra.cn/zh-cn/news/features/acl-2019-ming-zhou.

  17. 17.

    https://tech.sina.cn/2020-03-26/detail-iimxyqwa3399986.d.html.

  18. 18.

    https://cloud.tencent.com/developer/article/1586451.

  19. 19.

    https://www.leiphone.com/news/201804/vKH66rt5xW0dycQL.html.

  20. 20.

    https://www.leiphone.com/news/201812/Cy3HiAmUh6J7P3gB.html.

  21. 21.

    https://www.jiqizhixin.com/articles/2019-12-10-7.

  22. 22.

    https://zhidx.com/p/144766.html.

References

  • Abdel-Hamid, O. and Jiang, H. (2013). Rapid and effective speaker adaptation of convolutional neural network based models for speech recognition. In INTERSPEECH, pages 1248–1252.

    Google Scholar 

  • Ackermann, S., Schawinski, K., Zhang, C., Weigel, A. K., and Turp, M. D. (2018). Using transfer learning to detect galaxy mergers. Monthly Notices of the Royal Astronomical Society, 479(1):415–425.

    Article  Google Scholar 

  • Ahmed, U., Khan, A., Khan, S. H., Basit, A., Haq, I. U., and Lee, Y. S. (2019). Transfer learning and meta classification based deep churn prediction system for telecom industry. arXiv preprint arXiv:1901.06091.

    Google Scholar 

  • Appelgren, M., Schrempf, P., Falis, M., Ikeda, S., and O’Neil, A. Q. (2019). Language transfer for early warning of epidemics from social media. arXiv preprint arXiv:1910.04519.

    Google Scholar 

  • Baalouch, M., Defurne, M., Poli, J.-P., and Cherrier, N. (2019). Sim-to-real domain adaptation for high energy physics. arXiv preprint arXiv:1912.08001.

    Google Scholar 

  • Bai, L., Yao, L., Kanhere, S. S., Yang, Z., Chu, J., and Wang, X. (2019). Passenger demand forecasting with multi-task convolutional recurrent neural networks. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 29–42. Springer.

    Google Scholar 

  • Bengio, Y., Lecun, Y., and Hinton, G. (2021). Deep learning for AI. Communications of the ACM, 64(7):58–65.

    Article  Google Scholar 

  • Bertoldi, N. and Federico, M. (2009). Domain adaptation for statistical machine translation with monolingual resources. In Proceedings of the fourth workshop on statistical machine translation, pages 182–189.

    Google Scholar 

  • Bian, W., Tao, D., and Rui, Y. (2011). Cross-domain human action recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(2):298–307.

    Article  Google Scholar 

  • Blanchard, G., Lee, G., and Scott, C. (2011). Generalizing from several related classification tasks to a new unlabeled sample. Advances in neural information processing systems, 24.

    Google Scholar 

  • Blitzer, J., McDonald, R., and Pereira, F. (2006). Domain adaptation with structural correspondence learning. In EMNLP, pages 120–128.

    Google Scholar 

  • Boutsioukis, G., Partalas, I., and Vlahavas, I. (2011). Transfer learning in multi-agent reinforcement learning domains. In European Workshop on Reinforcement Learning, pages 249–260. Springer.

    Google Scholar 

  • Bray, C. W. (1928). Transfer of learning. Journal of Experimental Psychology, 11(6):443.

    Article  Google Scholar 

  • Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. In NeurIPS.

    Google Scholar 

  • Cabezas, M., Valverde, S., González-Villà, S., Clérigues, A., Salem, M., Kushibar, K., Bernal, J., Oliver, A., and Lladó, X. (2018). Survival prediction using ensemble tumor segmentation and transfer learning. arXiv preprint arXiv:1810.04274.

    Google Scholar 

  • Cao, H., Bernard, S., Heutte, L., and Sabourin, R. (2018). Improve the performance of transfer learning without fine-tuning using dissimilarity-based multi-view learning for breast cancer histology images. In International conference image analysis and recognition, pages 779–787. Springer.

    Google Scholar 

  • Carr, T., Chli, M., and Vogiatzis, G. (2018). Domain adaptation for reinforcement learning on the Atari. arXiv preprint arXiv:1812.07452.

    Google Scholar 

  • Chao, H., He, Y., Zhang, J., and Feng, J. (2019). GaitSet: Regarding gait as a set for cross-view gait recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8126–8133.

    Google Scholar 

  • Chen, B., Cherry, C., Foster, G., and Larkin, S. (2017a). Cost weighting for neural machine translation domain adaptation. In Proceedings of the First Workshop on Neural Machine Translation, pages 40–46.

    Google Scholar 

  • Chen, B. and Huang, F. (2016). Semi-supervised convolutional networks for translation adaptation with tiny amount of in-domain data. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 314–323.

    Google Scholar 

  • Chen, C., Dou, Q., Chen, H., and Heng, P.-A. (2018a). Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation. In International workshop on machine learning in medical imaging, pages 143–151. Springer.

    Google Scholar 

  • Chen, D., Ong, C. S., and Menon, A. K. (2019a). Cold-start playlist recommendation with multitask learning. arXiv preprint arXiv:1901.06125.

    Google Scholar 

  • Chen, H., Cui, S., and Li, S. (2017b). Application of transfer learning approaches in multimodal wearable human activity recognition. arXiv preprint arXiv:1707.02412.

    Google Scholar 

  • Chen, H., Wang, Y., Wang, G., and Qiao, Y. (2018b). LSTD: A low-shot transfer detector for object detection. In Thirty-Second AAAI Conference on Artificial Intelligence.

    Google Scholar 

  • Chen, L. (2018). Deep transfer learning for static malware classification. arXiv preprint arXiv:1812.07606.

    Google Scholar 

  • Chen, L.-W., Lee, H.-Y., and Tsao, Y. (2018c). Generative adversarial networks for unpaired voice transformation on impaired speech. arXiv preprint arXiv:1810.12656.

    Google Scholar 

  • Chen, S., Ma, K., and Zheng, Y. (2019b). Med3d: Transfer learning for 3d medical image analysis. arXiv preprint arXiv:1904.00625.

    Google Scholar 

  • Chen, Y., Assael, Y., Shillingford, B., Budden, D., Reed, S., Zen, H., Wang, Q., Cobo, L. C., Trask, A., Laurie, B., et al. (2018d). Sample efficient adaptive text-to-speech. arXiv preprint arXiv:1809.10460.

    Google Scholar 

  • Chen, Y., Li, W., Sakaridis, C., Dai, D., and Van Gool, L. (2018e). Domain adaptive faster R-CNN for object detection in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3339–3348.

    Google Scholar 

  • Chen, Y., Qin, X., Wang, J., Yu, C., and Gao, W. (2020). FedHealth: A federated transfer learning framework for wearable healthcare. IEEE Intelligent Systems, 35(4):83–93.

    Article  Google Scholar 

  • Chou, J.-c., Yeh, C.-c., and Lee, H.-y. (2019). One-shot voice conversion by separating speaker and content representations with instance normalization. arXiv preprint arXiv:1904.05742.

    Google Scholar 

  • Cooper, E., Lai, C.-I., Yasuda, Y., Fang, F., Wang, X., Chen, N., and Yamagishi, J. (2020). Zero-shot multi-speaker text-to-speech with state-of-the-art neural speaker embeddings. In ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6184–6188. IEEE.

    Google Scholar 

  • Covas, E. (2020). Transfer learning in spatial–temporal forecasting of the solar magnetic field. Astronomische Nachrichten.

    Google Scholar 

  • Cui, W., Zheng, G., Shen, Z., Jiang, S., and Wang, W. (2019). Transfer learning for sequences via learning to collocate. arXiv preprint arXiv:1902.09092.

    Google Scholar 

  • Daher, R., Zein, M. K., Zini, J. E., Awad, M., and Asmar, D. (2019). Change your singer: a transfer learning generative adversarial framework for song to song conversion. arXiv preprint arXiv:1911.02933.

    Google Scholar 

  • Dai, Q., Shen, X., Wu, X.-M., and Wang, D. (2019). Network transfer learning via adversarial domain adaptation with graph convolution. arXiv preprint arXiv:1909.01541.

    Google Scholar 

  • Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. IEEE.

    Google Scholar 

  • Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL.

    Google Scholar 

  • Diba, A., Fayyaz, M., Sharma, V., Karami, A. H., Arzani, M. M., Yousefzadeh, R., and Van Gool, L. (2017). Temporal 3D ConvNets: New architecture and transfer learning for video classification. arXiv preprint arXiv:1711.08200.

    Google Scholar 

  • Ding, M., Wang, Y., Hemberg, E., and O’Reilly, U.-M. (2019). Transfer learning using representation learning in massive open online courses. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge, pages 145–154.

    Chapter  Google Scholar 

  • Dou, Q., Ouyang, C., Chen, C., Chen, H., Glocker, B., Zhuang, X., and Heng, P.-A. (2018). PnP-AdaNet: Plug-and-play adversarial domain adaptation network with a benchmark at cross-modality cardiac segmentation. arXiv preprint arXiv:1812.07907.

    Google Scholar 

  • Feng, J., Huang, M., Zhao, L., Yang, Y., and Zhu, X. (2018). Reinforcement learning for relation classification from noisy data. In Thirty-Second AAAI Conference on Artificial Intelligence.

    Google Scholar 

  • Gamrian, S. and Goldberg, Y. (2019). Transfer learning for related reinforcement learning tasks via image-to-image translation. In International Conference on Machine Learning, pages 2063–2072.

    Google Scholar 

  • Gatys, L. A., Ecker, A. S., and Bethge, M. (2016). Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414–2423.

    Google Scholar 

  • Giacomello, E., Loiacono, D., and Mainardi, L. (2019). Transfer brain MRI tumor segmentation models across modalities with adversarial networks. arXiv preprint arXiv:1910.02717.

    Google Scholar 

  • Giel, A. and Diaz, R. (2015). Recurrent neural networks and transfer learning for action recognition.

    Google Scholar 

  • Goussies, N. A., Ubalde, S., Fernández, F. G., and Mejail, M. E. (2014). Optical character recognition using transfer learning decision forests. In 2014 IEEE International Conference on Image Processing (ICIP), pages 4309–4313. IEEE.

    Google Scholar 

  • Grave, E., Obozinski, G., and Bach, F. (2013). Domain adaptation for sequence labeling using hidden Markov models. arXiv preprint arXiv:1312.4092.

    Google Scholar 

  • Gupta, A., Devin, C., Liu, Y., Abbeel, P., and Levine, S. (2017). Learning invariant feature spaces to transfer skills with reinforcement learning. arXiv preprint arXiv:1703.02949.

    Google Scholar 

  • Gupta, P., Malhotra, P., Narwariya, J., Vig, L., and Shroff, G. (2020). Transfer learning for clinical time series analysis using deep neural networks. Journal of Healthcare Informatics Research, 4(2):112–137.

    Article  Google Scholar 

  • Gupta, S. and Raghavan, P. (2004). Adaptation of speech models in speech recognition. US Patent App. 10/447,906.

    Google Scholar 

  • Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A. (2020). Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964.

    Google Scholar 

  • He, H. and Wu, D. (2019). Transfer learning for brain–computer interfaces: A Euclidean space data alignment approach. IEEE Transactions on Biomedical Engineering, 67(2):399–410.

    Article  Google Scholar 

  • He, J., Lawrence, R. D., and Liu, Y. (2016). Graph-based transfer learning. US Patent 9,477,929.

    Google Scholar 

  • Hou, W., Wang, J., Tan, X., Qin, T., and Shinozaki, T. (2021). Cross-domain speech recognition with unsupervised character-level distribution matching. In Interspeech.

    Google Scholar 

  • Hou, W., Zhu, H., Wang, Y., Wang, J., Qin, T., Xu, R., and Shinozaki, T. (2022). Exploiting adapters for cross-lingual low-resource speech recognition. IEEE Transactions on Audio, Speech and Language Processing (TASLP).

    Google Scholar 

  • Hsu, W.-N., Zhang, Y., and Glass, J. (2017). Unsupervised domain adaptation for robust speech recognition via variational autoencoder-based data augmentation. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 16–23. IEEE.

    Google Scholar 

  • Hu, D. H. and Yang, Q. (2011). Transfer learning for activity recognition via sensor mapping. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence, volume 22, page 1962, Barcelona, Catalonia, Spain. IJCAI.

    Google Scholar 

  • Hu, D. H., Zheng, V. W., and Yang, Q. (2011). Cross-domain activity recognition via transfer learning. Pervasive and Mobile Computing, 7(3):344–358.

    Article  Google Scholar 

  • Hu, Q., Whitney, H. M., and Giger, M. L. (2019). Transfer learning in 4d for breast cancer diagnosis using dynamic contrast-enhanced magnetic resonance imaging. arXiv preprint arXiv:1911.03022.

    Google Scholar 

  • Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., Schölkopf, B., et al. (2007). Correcting sample selection bias by unlabeled data. Advances in neural information processing systems, 19:601.

    Google Scholar 

  • Huang, X. and Belongie, S. (2017). Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, pages 1501–1510.

    Google Scholar 

  • Huang, Z., Siniscalchi, S. M., and Lee, C.-H. (2016). A unified approach to transfer learning of deep neural networks with applications to speaker adaptation in automatic speech recognition. Neurocomputing, 218:448–459.

    Article  Google Scholar 

  • Humbird, K. D., Peterson, J. L., Spears, B., and McClarren, R. (2019). Transfer learning to model inertial confinement fusion experiments. IEEE Transactions on Plasma Science, 48(1):61–70.

    Article  Google Scholar 

  • Inoue, N., Furuta, R., Yamasaki, T., and Aizawa, K. (2018). Cross-domain weakly-supervised object detection through progressive domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5001–5009.

    Google Scholar 

  • Ionita, A., Pomp, A., Cochez, M., Meisen, T., and Decker, S. (2019). Transferring knowledge from monitored to unmonitored areas for forecasting parking spaces. International Journal on Artificial Intelligence Tools, 28(06):1960003.

    Article  Google Scholar 

  • Jia, C., Kong, Y., Ding, Z., and Fu, Y. R. (2014). Latent tensor transfer learning for RGB-D action recognition. In Proceedings of the 22nd ACM international conference on Multimedia, pages 87–96.

    Google Scholar 

  • Jia, Y., Zhang, Y., Weiss, R., Wang, Q., Shen, J., Ren, F., Nguyen, P., Pang, R., Moreno, I. L., Wu, Y., et al. (2018). Transfer learning from speaker verification to multispeaker text-to-speech synthesis. In Advances in neural information processing systems, pages 4480–4490.

    Google Scholar 

  • Jiang, J. and Zhai, C. (2007). Instance weighting for domain adaptation in nlp. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 264–271.

    Google Scholar 

  • Kachuee, M., Fazeli, S., and Sarrafzadeh, M. (2018). ECG heartbeat classification: A deep transferable representation. In 2018 IEEE International Conference on Healthcare Informatics (ICHI), pages 443–444. IEEE.

    Google Scholar 

  • Kamnitsas, K., Baumgartner, C., Ledig, C., Newcombe, V., Simpson, J., Kane, A., Menon, D., Nori, A., Criminisi, A., Rueckert, D., et al. (2017). Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In International conference on information processing in medical imaging, pages 597–609. Springer.

    Google Scholar 

  • Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018). Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 172(5):1122–1131.

    Article  Google Scholar 

  • Khan, M. A. A. H. and Roy, N. (2017). Transact: Transfer learning enabled activity recognition. In 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pages 545–550. IEEE.

    Google Scholar 

  • Kim, M., Kim, Y., Yoo, J., Wang, J., and Kim, H. (2017). Regularized speaker adaptation of KL-HMM for dysarthric speech recognition. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(9):1581–1591.

    Article  Google Scholar 

  • Lee, J., Kim, H., Lee, J., and Yoon, S. (2017). Transfer learning for deep learning on graph-structured data. In AAAI, pages 2154–2160.

    Google Scholar 

  • Li, B., Wang, X., and Beigi, H. (2019a). Cantonese automatic speech recognition using transfer learning from mandarin. arXiv preprint arXiv:1911.09271.

    Google Scholar 

  • Li, P., Lou, P., Yan, J., and Liu, N. (2020). The thermal error modeling with deep transfer learning. In Journal of Physics: Conference Series, volume 1576, page 012003. IOP Publishing.

    Google Scholar 

  • Li, X., Chen, Y., Wu, Z., Peng, X., Wang, J., Hu, L., and Yu, D. (2017a). Weak multipath effect identification for indoor distance estimation. In UIC, pages 1–8. IEEE.

    Google Scholar 

  • Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., and Yang, M.-H. (2017b). Universal style transfer via feature transforms. In Advances in neural information processing systems, pages 386–396.

    Google Scholar 

  • Li, Y., Yuan, L., and Vasconcelos, N. (2019b). Bidirectional learning for domain adaptation of semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6936–6945.

    Google Scholar 

  • Liao, H. (2013). Speaker adaptation of context dependent deep neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7947–7951. IEEE.

    Google Scholar 

  • Lim, J. J., Salakhutdinov, R. R., and Torralba, A. (2011). Transfer learning by borrowing examples for multiclass object detection. In Advances in neural information processing systems, pages 118–126.

    Google Scholar 

  • Liu, J., Chen, Y., and Zhang, Y. (2010). Transfer regression model for indoor 3d location estimation. In International Conference on Multimedia Modeling, pages 603–613. Springer.

    Google Scholar 

  • Liu, J., Shah, M., Kuipers, B., and Savarese, S. (2011). Cross-view action recognition via view knowledge transfer. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3209–3216, Colorado Springs, CO, USA. IEEE.

    Google Scholar 

  • Liu, M., Song, Y., Zou, H., and Zhang, T. (2019). Reinforced training data selection for domain adaptation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1957–1968.

    Google Scholar 

  • Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2021). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586.

    Google Scholar 

  • Liu, S., Zhong, J., Sun, L., Wu, X., Liu, X., and Meng, H. (2018). Voice conversion across arbitrary speakers based on a single target-speaker utterance. In Interspeech, pages 496–500.

    Google Scholar 

  • Luan, F., Paris, S., Shechtman, E., and Bala, K. (2017). Deep photo style transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4990–4998.

    Google Scholar 

  • Luo, Y., Zheng, L., Guan, T., Yu, J., and Yang, Y. (2019). Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2507–2516.

    Google Scholar 

  • Mallick, T., Balaprakash, P., Rask, E., and Macfarlane, J. (2020). Transfer learning with graph neural networks for short-term highway traffic forecasting. arXiv preprint arXiv:2004.08038.

    Google Scholar 

  • Manakov, I., Rohm, M., Kern, C., Schworm, B., Kortuem, K., and Tresp, V. (2019). Noise as domain shift: Denoising medical images by unpaired image translation. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data, pages 3–10. Springer.

    Google Scholar 

  • Mari, A., Bromley, T. R., Izaac, J., Schuld, M., and Killoran, N. (2019). Transfer learning in hybrid classical-quantum neural networks. arXiv preprint arXiv:1912.08278.

    Google Scholar 

  • Marinescu, R. V., Lorenzi, M., Blumberg, S. B., Young, A. L., Planell-Morell, P., Oxtoby, N. P., Eshaghi, A., Yong, K. X., Crutch, S. J., Golland, P., et al. (2019). Disease knowledge transfer across neurodegenerative diseases. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 860–868. Springer.

    Google Scholar 

  • McClosky, D., Charniak, E., and Johnson, M. (2010). Automatic domain adaptation for parsing. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 28–36. Association for Computational Linguistics.

    Google Scholar 

  • Milhomem, S., Almeida, T. d. S., da Silva, W. G., da Silva, E. M., and de Carvalho, R. L. (2019). Weightless neural network with transfer learning to detect distress in asphalt. arXiv preprint arXiv:1901.03660.

    Google Scholar 

  • Morales, F. J. O. and Roggen, D. (2016). Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations. In Proceedings of the 2016 ACM International Symposium on Wearable Computers, pages 92–99.

    Google Scholar 

  • Newman-Griffis, D. and Zirikly, A. (2018). Embedding transfer for low-resource medical named entity recognition: a case study on patient mobility. arXiv preprint arXiv:1806.02814.

    Google Scholar 

  • Nguyen, D., Nguyen, K., Sridharan, S., Abbasnejad, I., Dean, D., and Fookes, C. (2018). Meta transfer learning for facial emotion recognition. In 2018 24th International Conference on Pattern Recognition (ICPR), pages 3543–3548. IEEE.

    Google Scholar 

  • Nguyen, D., Sridharan, S., Nguyen, D. T., Denman, S., Tran, S. N., Zeng, R., and Fookes, C. (2020). Joint deep cross-domain transfer learning for emotion recognition. arXiv preprint arXiv:2003.11136.

    Google Scholar 

  • Nguyen, L. H., Zhu, J., Lin, Z., Du, H., Yang, Z., Guo, W., and Jin, F. (2019). Spatial-temporal multi-task learning for within-field cotton yield prediction. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 343–354. Springer.

    Google Scholar 

  • Oliveira, J. S., Souza, G. B., Rocha, A. R., Deus, F. E., and Marana, A. N. (2020). Cross-domain deep face matching for real banking security systems. In 2020 Seventh International Conference on eDemocracy & eGovernment (ICEDEG), pages 21–28. IEEE.

    Google Scholar 

  • Omran, P. G., Wang, Z., and Wang, K. (2019). Knowledge graph rule mining via transfer learning. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 489–500. Springer.

    Google Scholar 

  • Pan, S. J., Kwok, J. T., and Yang, Q. (2008). Transfer learning via dimensionality reduction. In Proceedings of the 23rd AAAI conference on Artificial intelligence, volume 8, pages 677–682.

    Google Scholar 

  • Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q. (2011). Domain adaptation via transfer component analysis. IEEE TNN, 22(2):199–210.

    Google Scholar 

  • Pan, S. J. and Yang, Q. (2010). A survey on transfer learning. IEEE TKDE, 22(10):1345–1359.

    Google Scholar 

  • Parisotto, E., Ba, J. L., and Salakhutdinov, R. (2015). Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv preprint arXiv:1511.06342.

    Google Scholar 

  • Peng, N. and Dredze, M. (2016). Multi-task domain adaptation for sequence tagging. arXiv preprint arXiv:1608.02689.

    Google Scholar 

  • Perone, C. S., Ballester, P., Barros, R. C., and Cohen-Adad, J. (2019). Unsupervised domain adaptation for medical imaging segmentation with self-ensembling. NeuroImage, 194:1–11.

    Article  Google Scholar 

  • Phan, H., Chén, O. Y., Koch, P., Mertins, A., and De Vos, M. (2019). Deep transfer learning for single-channel automatic sleep staging with channel mismatch. In 2019 27th European Signal Processing Conference (EUSIPCO), pages 1–5. IEEE.

    Google Scholar 

  • Poncelas, A., Wenniger, G. M. d. B., and Way, A. (2019). Transductive data-selection algorithms for fine-tuning neural machine translation. arXiv preprint arXiv:1908.09532.

    Google Scholar 

  • Prodanova, N., Stegmaier, J., Allgeier, S., Bohn, S., Stachs, O., Köhler, B., Mikut, R., and Bartschat, A. (2018). Transfer learning with human corneal tissues: An analysis of optimal cut-off layer. arXiv preprint arXiv:1806.07073.

    Google Scholar 

  • Qu, C., Ji, F., Qiu, M., Yang, L., Min, Z., Chen, H., Huang, J., and Croft, W. B. (2019). Learning to selectively transfer: Reinforced transfer learning for deep text matching. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 699–707.

    Chapter  Google Scholar 

  • Quiñonero-Candela, J., Sugiyama, M., Lawrence, N. D., and Schwaighofer, A. (2009). Dataset shift in machine learning. MIT Press.

    Google Scholar 

  • Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving language understanding by generative pre-training.

    Google Scholar 

  • Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.

    Google Scholar 

  • Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683.

    Google Scholar 

  • Rahmani, H. and Mian, A. (2015). Learning a non-linear knowledge transfer model for cross-view action recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2458–2466.

    Google Scholar 

  • Raj, A., Namboodiri, V. P., and Tuytelaars, T. (2015). Subspace alignment based domain adaptation for RCNN detector. arXiv preprint arXiv:1507.05578.

    Google Scholar 

  • Rathi, D. (2018). Optimization of transfer learning for sign language recognition targeting mobile platform. arXiv preprint arXiv:1805.06618.

    Google Scholar 

  • Rehman, N. A., Aliapoulios, M. M., Umarwani, D., and Chunara, R. (2018). Domain adaptation for infection prediction from symptoms based on data from different study designs and contexts. arXiv preprint arXiv:1806.08835.

    Google Scholar 

  • Ren, J., Hacihaliloglu, I., Singer, E. A., Foran, D. J., and Qi, X. (2018). Adversarial domain adaptation for classification of prostate histopathology whole-slide images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 201–209. Springer.

    Google Scholar 

  • Rezaei, M., Yang, H., and Meinel, C. (2018). Multi-task generative adversarial network for handling imbalanced clinical data. arXiv preprint arXiv:1811.10419.

    Google Scholar 

  • Ruder, S. and Plank, B. (2017). Learning to select data for transfer learning with Bayesian optimization. arXiv preprint arXiv:1707.05246.

    Google Scholar 

  • Saito, Y. (2019). Unsupervised domain adaptation meets offline recommender learning. arXiv preprint arXiv:1910.07295.

    Google Scholar 

  • Salem, M., Taheri, S., and Yuan, J.-S. (2018). ECG arrhythmia classification using transfer learning from 2-dimensional deep CNN features. In 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), pages 1–4. IEEE.

    Google Scholar 

  • Sankaranarayanan, S., Balaji, Y., Jain, A., Lim, S. N., and Chellappa, R. (2017). Unsupervised domain adaptation for semantic segmentation with GANs. arXiv preprint arXiv:1711.06969, 2:2.

    Google Scholar 

  • Sargano, A. B., Wang, X., Angelov, P., and Habib, Z. (2017). Human action recognition using transfer learning with deep representations. In 2017 International joint conference on neural networks (IJCNN), pages 463–469. IEEE.

    Google Scholar 

  • Shi, Z., Siva, P., and Xiang, T. (2017). Transfer learning by ranking for weakly supervised object annotation. arXiv preprint arXiv:1705.00873.

    Google Scholar 

  • Shivakumar, P. G., Potamianos, A., Lee, S., and Narayanan, S. S. (2014). Improving speech recognition for children using acoustic adaptation and pronunciation modeling. In WOCCI, pages 15–19.

    Google Scholar 

  • Sun, B. and Saenko, K. (2014). From virtual to reality: Fast adaptation of virtual object detectors to real domains. In BMVC, volume 1, page 3.

    Google Scholar 

  • Sun, L., Li, K., Wang, H., Kang, S., and Meng, H. (2016). Phonetic posteriorgrams for many-to-one voice conversion without parallel data training. In 2016 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE.

    Google Scholar 

  • Sun, S., Zhang, B., Xie, L., and Zhang, Y. (2017). An unsupervised deep domain adaptation approach for robust speech recognition. Neurocomputing, 257:79–87.

    Article  Google Scholar 

  • Sun, X. and Wei, J. (2020). Identification of maize disease based on transfer learning. In Journal of Physics: Conference Series, volume 1437, page 012080. IOP Publishing.

    Google Scholar 

  • Sun, Z., Chen, Y., Qi, J., and Liu, J. (2008). Adaptive localization through transfer learning in indoor Wi-Fi environment. In 2008 Seventh International Conference on Machine Learning and Applications, pages 331–336. IEEE.

    Google Scholar 

  • Suresh, H., Gong, J. J., and Guttag, J. V. (2018). Learning tasks for multitask learning: Heterogenous patient populations in the ICU. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 802–810.

    Google Scholar 

  • Tang, X., Li, Y., Sun, Y., Yao, H., Mitra, P., and Wang, S. (2019). Robust graph neural network against poisoning attacks via transfer learning. arXiv preprint arXiv:1908.07558.

    Google Scholar 

  • Tang, Y., Peng, L., Xu, Q., Wang, Y., and Furuhata, A. (2016). CNN based transfer learning for historical Chinese character recognition. In 2016 12th IAPR Workshop on Document Analysis Systems (DAS), pages 25–29. IEEE.

    Google Scholar 

  • Taylor, M. E. and Stone, P. (2009). Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(Jul):1633–1685.

    MathSciNet  MATH  Google Scholar 

  • Tsai, Y.-H., Hung, W.-C., Schulter, S., Sohn, K., Yang, M.-H., and Chandraker, M. (2018). Learning to adapt structured output space for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7472–7481.

    Google Scholar 

  • Tu, G., Fu, Y., Li, B., Gao, J., Jiang, Y.-G., and Xue, X. (2019). A multi-task neural approach for emotion attribution, classification, and summarization. IEEE Transactions on Multimedia, 22(1):148–159.

    Article  Google Scholar 

  • Tzu, C. (2006). The Book of Chuang Tzu. Penguin UK.

    Google Scholar 

  • Valverde, S., Salem, M., Cabezas, M., Pareto, D., Vilanova, J. C., Ramió-Torrentà, L., Rovira, À., Salvi, J., Oliver, A., and Lladó, X. (2019). One-shot domain adaptation in multiple sclerosis lesion segmentation using convolutional neural networks. NeuroImage: Clinical, 21:101638.

    Article  Google Scholar 

  • Vanschoren, J. (2018). Meta-learning: A survey. arXiv preprint arXiv:1810.03548.

    Google Scholar 

  • Venkataramani, R., Ravishankar, H., and Anamandra, S. (2018). Towards continuous domain adaptation for healthcare. arXiv preprint arXiv:1812.01281.

    Google Scholar 

  • Venkateswara, H., Eusebio, J., Chakraborty, S., and Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018–5027.

    Google Scholar 

  • Vilalta, R., Gupta, K. D., Boumber, D., and Meskhi, M. M. (2019). A general approach to domain adaptation with applications in astronomy. Publications of the Astronomical Society of the Pacific, 131(1004):108008.

    Article  Google Scholar 

  • Waley, A. et al. (2005). The analects of Confucius, volume 28. Psychology Press.

    Google Scholar 

  • Wang, J., Chen, Y., Hu, L., Peng, X., and Yu, P. S. (2018a). Stratified transfer learning for cross-domain activity recognition. In 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

    Google Scholar 

  • Wang, J., Lan, C., Liu, C., Ouyang, Y., Zeng, W., and Qin, T. (2021). Generalizing to unseen domains: A survey on domain generalization. In IJCAI Survey Track.

    Google Scholar 

  • Wang, J., Zheng, V. W., Chen, Y., and Huang, M. (2018b). Deep transfer learning for cross-domain activity recognition. In proceedings of the 3rd International Conference on Crowd Science and Engineering, pages 1–8.

    Google Scholar 

  • Wang, R., Utiyama, M., Liu, L., Chen, K., and Sumita, E. (2017). Instance weighting for neural machine translation domain adaptation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1482–1488.

    Google Scholar 

  • Wang, Z., Bi, W., Wang, Y., and Liu, X. (2019). Better fine-tuning via instance weighting for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7241–7248.

    Google Scholar 

  • Weiss, K., Khoshgoftaar, T. M., and Wang, D. (2016). A survey of transfer learning. Journal of Big data, 3(1):1–40.

    Article  Google Scholar 

  • Wenzel, P., Khan, Q., Cremers, D., and Leal-Taixé, L. (2018). Modular vehicle control for transferring semantic information between weather conditions using GANs. arXiv preprint arXiv:1807.01001.

    Google Scholar 

  • Woodworth, R. S. and Thorndike, E. (1901). The influence of improvement in one mental function upon the efficiency of other functions.(i). Psychological review, 8(3):247.

    Google Scholar 

  • Wu, C. and Gales, M. J. (2015). Multi-basis adaptive neural network for rapid adaptation in speech recognition. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4315–4319. IEEE.

    Google Scholar 

  • Wu, F. and Huang, Y. (2016). Sentiment domain adaptation with multiple sources. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 301–310.

    Google Scholar 

  • Wu, X., Wang, H., Liu, C., and Jia, Y. (2013). Cross-view action recognition over heterogeneous feature spaces. In Proceedings of the IEEE International Conference on Computer Vision, pages 609–616.

    Google Scholar 

  • Xie, M., Jean, N., Burke, M., Lobell, D., and Ermon, S. (2016). Transfer learning from deep features for remote sensing and poverty mapping. In Thirtieth AAAI Conference on Artificial Intelligence.

    Google Scholar 

  • Xu, N., Zheng, G., Xu, K., Zhu, Y., and Li, Z. (2019). Targeted knowledge transfer for learning traffic signal plans. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 175–187. Springer.

    Google Scholar 

  • Xue, S., Abdel-Hamid, O., Jiang, H., Dai, L., and Liu, Q. (2014). Fast adaptation of deep neural network based on discriminant codes for speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(12):1713–1725.

    Article  Google Scholar 

  • Yang, Z., Salakhutdinov, R., and Cohen, W. W. (2017). Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv preprint arXiv:1703.06345.

    Google Scholar 

  • Yao, K., Yu, D., Seide, F., Su, H., Deng, L., and Gong, Y. (2012). Adaptation of context-dependent deep neural networks for automatic speech recognition. In 2012 IEEE Spoken Language Technology Workshop (SLT), pages 366–369. IEEE.

    Google Scholar 

  • Ye, Z., Yang, Y., Li, X., Cao, D., and Ouyang, D. (2018). An integrated transfer learning and multitask learning approach for pharmacokinetic parameter prediction. Molecular pharmaceutics, 16(2):533–541.

    Article  Google Scholar 

  • Yu, C., Wang, J., Liu, C., Qin, T., Xu, R., Feng, W., Chen, Y., and Liu, T.-Y. (2020). Learning to match distributions for domain adaptation. arXiv preprint arXiv:2007.10791.

    Google Scholar 

  • Yu, D., Yao, K., Su, H., Li, G., and Seide, F. (2013). KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7893–7897. IEEE.

    Google Scholar 

  • Yu, F., Zhao, J., Gong, Y., Wang, Z., Li, Y., Yang, F., Dong, B., Li, Q., and Zhang, L. (2019). Annotation-free cardiac vessel segmentation via knowledge transfer from retinal images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 714–722. Springer.

    Google Scholar 

  • Yu, T., Mutter, D., Marescaux, J., and Padoy, N. (2018). Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition. arXiv preprint arXiv:1812.00033.

    Google Scholar 

  • Zamir, A. R., Sax, A., Shen, W., Guibas, L. J., Malik, J., and Savarese, S. (2018). Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3712–3722.

    Google Scholar 

  • Zhang, C. and Peng, Y. (2018). Better and faster: knowledge transfer from multiple self-supervised learning tasks via graph distillation for video classification. arXiv preprint arXiv:1804.10069.

    Google Scholar 

  • Zhang, H., Chen, W., He, H., and Jin, Y. (2019a). Disentangled makeup transfer with generative adversarial network. arXiv preprint arXiv:1907.01144.

    Google Scholar 

  • Zhang, Y., David, P., and Gong, B. (2017). Curriculum domain adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE International Conference on Computer Vision, pages 2020–2030.

    Google Scholar 

  • Zhang, Y., Nie, S., Liu, W., Xu, X., Zhang, D., and Shen, H. T. (2019b). Sequence-to-sequence domain adaptation network for robust text image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2740–2749.

    Google Scholar 

  • Zhang, Y., Niu, S., Qiu, Z., Wei, Y., Zhao, P., Yao, J., Huang, J., Wu, Q., and Tan, M. (2020). COVID-DA: Deep domain adaptation from typical pneumonia to covid-19. arXiv preprint arXiv:2005.01577.

    Google Scholar 

  • Zhang, Y., Zhang, Y., and Yang, Q. (2019c). Parameter transfer unit for deep neural networks. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD).

    Google Scholar 

  • Zhao, Z., Chen, Y., Liu, J., Shen, Z., and Liu, M. (2011). Cross-people mobile-phone based activity recognition. In Proceedings of the Twenty-Second international joint conference on Artificial Intelligence (IJCAI), volume 11, pages 2545–2550. Citeseer.

    Google Scholar 

  • Zheng, J., Jiang, Z., and Chellappa, R. (2016). Cross-view action recognition via transferable dictionary learning. IEEE Transactions on Image Processing, 25(6):2542–2556.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu, Y., Xi, D., Song, B., Zhuang, F., Chen, S., Gu, X., and He, Q. (2020). Modeling users’ behavior sequences with hierarchical explainable network for cross-domain fraud detection. In Proceedings of The Web Conference 2020, pages 928–938.

    Google Scholar 

  • Zou, H., Zhou, Y., Jiang, H., Huang, B., Xie, L., and Spanos, C. (2017). Adaptive localization in dynamic indoor environments by transfer kernel learning. In 2017 IEEE wireless communications and networking conference (WCNC), pages 1–6. IEEE.

    Google Scholar 

  • Zou, Y., Yu, Z., Vijaya Kumar, B., and Wang, J. (2018). Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision (ECCV), pages 289–305.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research Asia (China), Beijing, China

    Jindong Wang

  2. Institute of Computing Technology, Beijing, China

    Yiqiang Chen

Authors
  1. Jindong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, J., Chen, Y. (2023). Introduction. In: Introduction to Transfer Learning. Machine Learning: Foundations, Methodologies, and Applications. Springer, Singapore. https://doi.org/10.1007/978-981-19-7584-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-7584-4_1

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-7583-7

  • Online ISBN: 978-981-19-7584-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation