Advertisement

An Overview of Transfer Learning Focused on Asymmetric Heterogeneous Approaches

  • Magda FriedjungováEmail author
  • Marcel Jiřina
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 814)

Abstract

In practice we often encounter classification tasks. In order to solve these tasks, we need a sufficient amount of quality data for the construction of an accurate classification model. However, in some cases, the collection of quality data poses a demanding challenge in terms of time and finances. For example in the medical area, we encounter lack of data about patients. Transfer learning introduces the idea that a possible solution can be combining data from different domains represented by different feature spaces relating to the same task. We can also transfer knowledge from a different but related task that has been learned already. This overview focuses on the current progress in the novel area of asymmetric heterogeneous transfer learning. We discuss approaches and methods for solving these types of transfer learning tasks. Furthermore, we mention the most used metrics and the possibility of using metric or similarity learning.

Keywords

Asymmetric heterogeneous transfer learning Different feature space Domain adaptation Survey Data mining Metric learning 

Notes

Acknowledgements

This research has been supported by SGS grant No. SGS17/210/OHK3/3T/18.

References

  1. 1.
    Friedjungová, M., Jiřina, M.: Asymmetric heterogeneous transfer learning: a survey. In: Proceedings of the 6th International Conference on Data Science, Technology and Applications - Volume 1: DATA, INSTICC, pp. 17–27. SciTePress (2017)Google Scholar
  2. 2.
    Zheng, Y.: Methodologies for cross-domain data fusion: an overview. IEEE Trans. Big Data 1(1), 16–34 (2015)CrossRefGoogle Scholar
  3. 3.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)CrossRefGoogle Scholar
  4. 4.
    Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big Data 3 (2016). https://doi.org/10.1186/s40537-016-0043-6. 9 pages
  5. 5.
    Daumé III, H.: Frustratingly easy domain adaptation (2009)Google Scholar
  6. 6.
    Bleiholder, J., Naumann, F.: Data fusion. ACM Comput. Surv. (CSUR) 41(1), 1 (2009)CrossRefGoogle Scholar
  7. 7.
    Cook, D.J., Feuz, K.D., Krishnan, N.C.: Transfer learning for activity recognition: a survey. Knowl. Inf. Syst. 36(3), 537–556 (2013)CrossRefGoogle Scholar
  8. 8.
    Feuz, K.D., Cook, D.J.: Transfer learning across feature-rich heterogeneous feature spaces via feature-space remapping (FSR). ACM Trans. Intell. Syst. Technol. 6(1) (2015). https://www.ncbi.nlm.nih.gov/pmc/utils/ctxp/?ids=PMC4804893&report=citeproc&format=json. 3 pagesCrossRefGoogle Scholar
  9. 9.
    Shi, X., Liu, Q., Fan, W., Wu, P.S., Zhu, R.: Transfer learning on heterogeneous feature spaces via spectral transformation. In: 2010 IEEE 10th International Conference on Data Mining (ICDM). IEEE (2010)Google Scholar
  10. 10.
    Prettenhofer, P., Stein, B.: Cross-language text classification using structural correspondence learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (2010)Google Scholar
  11. 11.
    Wang, C., Mahadevan, S.: Heterogeneous domain adaptation using manifold alignment. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence (2011)Google Scholar
  12. 12.
    Duan, L., Xu, D., Tsang, I.W.: Learning with augmented features for heterogeneous domain adaptation (2012)Google Scholar
  13. 13.
    Blitzer, S.J., McDonald, R., Pereira, F.: Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2006)Google Scholar
  14. 14.
    Ando, R.K., Zhang, T.: A high-performance semi-supervised learning method for text chunking. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL 2005, Stroudsburg, PA, USA, pp. 1–9. Association for Computational Linguistics (2005)Google Scholar
  15. 15.
    Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the 23rd National Conference on Artificial Intelligence (2008)Google Scholar
  16. 16.
    Daumé III, H., Kumar, A., Saha, A.: Frustratingly easy semi-supervised domain adaptation. In: Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing. Association for Computational Linguistics (2010)Google Scholar
  17. 17.
    Zhong, E., Fan, W., Peng, J., Zhang, K., Ren, J., Turaga, D., Verscheure, O.: Cross domain distribution adaptation via kernel mapping. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2009)Google Scholar
  18. 18.
    Kulis, B., Saenko, K., Darrell, T.: What you saw is not what you get: domain adaptation using asymmetric Kernel transforms. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2011)Google Scholar
  19. 19.
    Dai, W., Chen, Y., Xue, G.R., Yang, Q., Yu, Y.: Translated learning: transfer learning across different feature spaces. In: Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems (2008)Google Scholar
  20. 20.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 213–226. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15561-1_16CrossRefGoogle Scholar
  21. 21.
    Kulis, B., et al.: Metric learning: a survey. In: Foundations and Trends® in Machine Learning, Now Publishers, Inc. (2013)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)CrossRefGoogle Scholar
  23. 23.
    Wei, B., Pal, C.: Cross lingual adaptation: an experiment on sentiment classifications. In: Proceedings of the ACL 2010 Conference Short Papers (2010)Google Scholar
  24. 24.
    Zhou, J.T., Pan, S.J., Tsang, I.W., Yan, Y.: Hybrid heterogeneous transfer learning through deep learning. In: 28th AAAI Conference on Artificial Intelligence (2014)Google Scholar
  25. 25.
    Lafferty, J., Zhai, C.: Document language models, query models, and risk minimization for information retrieval. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM (2001)Google Scholar
  26. 26.
    Dai, W., Chen, Y., Xue, G., Yang, Q., Yu, Y.: Translated learning: transfer learning across different feature spaces. In: Advances in Neural Information Processing Systems 21, pp. 353–360. Curran Associates, Inc. (2009)Google Scholar
  27. 27.
    Zhou, J.T., Tsang, I.W., Sinno, P.J., Tan, M.: Heterogeneous domain adaptation for multiple classes. In: International Conference on Artificial Intelligence and Statistics (2014)Google Scholar
  28. 28.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)zbMATHGoogle Scholar
  29. 29.
    Ando, R.K., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res. 6, 1817–1853 (2005)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Nam, J., Pan, S.J., Kim, S.: Transfer defect learning. In: Proceedings of the 2013 International Conference on Software Engineering. IEEE (2013)Google Scholar
  31. 31.
    He, Z., Shu, F., Yang, Y., Li, M., Wang, Q.: An investigation on the feasibility of cross-project defect prediction. Autom. Softw. Eng. 19(2), 167–199 (2012)CrossRefGoogle Scholar
  32. 32.
    Ma, Y., Luo, G., Zeng, X., Chen, A.: Transfer learning for cross-company software defect prediction. Inf. Softw. Technol. 54(3), 248–256 (2012)CrossRefGoogle Scholar
  33. 33.
    Rahman, F., Posnett, D., Devanbu, P.: Recalling the “imprecision” of cross-project defect prediction. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM (2012)Google Scholar
  34. 34.
    Nam, J., Kim, S.: Heterogeneous defect prediction. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM (2015)Google Scholar
  35. 35.
    Rashidi, P., Cook, D.J.: Multi home transfer learning for resident activity discovery and recognition (2010)Google Scholar
  36. 36.
    van Kasteren, T.L.M., Englebienne, G., Kröse, B.J.A.: Transferring knowledge of activity recognition across sensor networks. In: Floréen, P., Krüger, A., Spasojevic, M. (eds.) Pervasive 2010. LNCS, vol. 6030, pp. 283–300. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-12654-3_17CrossRefGoogle Scholar
  37. 37.
    Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: 2010 International Symposium on Wearable Computers. IEEE (2010)Google Scholar
  38. 38.
    Harel, M., Mannor, S.: Learning from multiple outlooks. In: Proceedings of the 28th International Conference on Machine Learning (2011)Google Scholar
  39. 39.
    Nomizu, K., Sasaki, T.: Affine Differential Geometry, 1st edn. Cambridge University Press, Cambridge (1995)zbMATHGoogle Scholar
  40. 40.
    van der Vaart, A.W.: Asymptotic Statistics, 1st edn. Cambridge University Press, Cambridge (2000)Google Scholar
  41. 41.
    Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Wortman, J.: Learning bounds for domain adaptation. In: Advances in Neural Information Processing Systems 21. MIT Press (2007)Google Scholar
  42. 42.
    Cook, D.J., Krishnan, N.C.: Activity Learning, Discovering, Recognizing, and Predicting Human Behavior from Sensor Data. Wiley, New York (2015)Google Scholar
  43. 43.
    van Kasteren, T.L.M., Englebienne, G., Krose, B.J.A.: Recognizing activities in multiple contexts using transfer learning. In: AAAI Fall Symposium: AI in Eldercare: New Solutions to Old Problems (2008)Google Scholar
  44. 44.
    Si, S., Tao, D., Geng, B.: Bregman divergence-based regularization for transfer subspace learning. IEEE Trans. Knowl. Data Eng. 22(7), 929–942 (2010)CrossRefGoogle Scholar
  45. 45.
    Dai, W., Jin, O., Xue, G.R., Yang, Q., Yu, Y.: EigenTransfer: a unified framework for transfer learning. In: Proceedings of the 26th Annual International Conference on Machine Learning. ACM (2009)Google Scholar
  46. 46.
    Satpal, S., Sarawagi, S.: Domain adaptation of conditional probability models via feature subsetting. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 224–235. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-74976-9_23CrossRefGoogle Scholar
  47. 47.
    Bellet, A., Habrard, A., Sebban, M.: A survey on metric learning for feature vectors and structured data (2013)Google Scholar
  48. 48.
    Yang, L.: Distance metric learning: a comprehensive survey (2006)Google Scholar
  49. 49.
    Wang, H., Wang, W., Zhang, C., Xu, F.: Cross-domain metric learning based on information theory. In: 28th AAAI Conference on Artificial Intelligence (2014)Google Scholar
  50. 50.
    Chechik, G., Sharma, V., Shalit, U., Bengio, S.: Large scale online learning of image similarity through ranking. J. Mach. Learn. Res. 11, 1109–1135 (2010)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Kulis, B., Jain, P., Grauman, K.: Fast similarity search for learned metrics. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2143–2157 (2009)CrossRefGoogle Scholar
  52. 52.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE (2005)Google Scholar
  53. 53.
    Hu, J., Lu, J., Tan, Y.P., Zhou, J.: Deep transfer metric learning. IEEE Trans. Image Process. 25(12), 5576–5588 (2016)MathSciNetCrossRefGoogle Scholar
  54. 54.
    Wang, S., Jin, R.: An information geometry approach for distance metric learning. In: Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (2009)Google Scholar
  55. 55.
    Geng, B., Tao, D., Xu, C.: DAML: domain adaptation metric learning. IEEE Trans. Image Process. 20(10), 2980–2989 (2011)MathSciNetCrossRefGoogle Scholar
  56. 56.
    Zhang, Y., Yeung, D.Y.: Transfer metric learning by learning task relationships. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2010)Google Scholar
  57. 57.
    Day, O., Khoshgoftaar, T.M.: A survey on heterogeneous transfer learning. J. Big Data 4(1), 29 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Information TechnologyCzech Technical University in PraguePragueCzech Republic

Personalised recommendations