Domain Generalization Based on Transfer Component Analysis

  • Thomas Grubinger
  • Adriana Birlutiu
  • Holger Schöner
  • Thomas Natschläger
  • Tom Heskes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9094)

Abstract

This paper investigates domain generalization: How to use knowledge acquired from related domains and apply it to new domains? Transfer Component Analysis (TCA) learns a shared subspace by minimizing the dissimilarities across domains, while maximally preserving the data variance. We propose Multi-TCA, an extension of TCA to multiple domains as well as Multi-SSTCA, which is an extension of TCA for semi-supervised learning. In addition to the original application of TCA for domain adaptation problems, we show that Multi-TCA can also be applied for domain generalization. Multi-TCA and Multi-SSTCA are evaluated on two publicly available datasets with the tasks of landmine detection and Parkinson telemonitoring. Experimental results demonstrate that Multi-TCA can improve predictive performance on previously unseen domains.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belkin, M.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)MATHMathSciNetGoogle Scholar
  2. 2.
    Bickel, S., Brückner, M., Scheffer, T.: Discriminative learning under covariate shift. J. Mach. Learn. Res. 10, 2137–2155 (2009)MATHMathSciNetGoogle Scholar
  3. 3.
    Blanchard, G., Lee, G., Scott, C.: Generalizing from several related classification tasks to a new unlabeled sample. In: NIPS, pp. 2178–2186 (2011)Google Scholar
  4. 4.
    Gong, B., Grauman, K., Sha, F.: Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In: ICML, pp. 222–230 (2013)Google Scholar
  5. 5.
    Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A.: A kernel method for the two-sample-problem. In: NIPS, pp. 513–520 (2006)Google Scholar
  6. 6.
    Gretton, A., Bousquet, O., Smola, A.J., Schölkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 63–77. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  7. 7.
    Hinton, G.E., Salakhutdinov, R.: Using deep belief nets to learn covariance kernels for gaussian processes. In: NIPS, pp. 1249–1256 (2007)Google Scholar
  8. 8.
    Huang, J., Smola, A.J., Gretton, A., Borgwardt, K.M., Schölkopf, B.: Correcting sample selection bias by unlabeled data. In: NIPS, pp. 601–608 (2006)Google Scholar
  9. 9.
    Little, M., McSharry, P., Roberts, S., Costello, D., Moroz, I.: Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. BioMedical Engineering OnLine 6:23(1) (2007)Google Scholar
  10. 10.
    Long, M., Pan, S.J., St Yu, P., Wang, J., Ding, G.: Adaptation regularization: A general framework for transfer learning. IEEE Trans. on Know. and Data Eng. 26(5), 1076–1089 (2014)CrossRefGoogle Scholar
  11. 11.
    Long, M., Wang, J., Ding, G., Shen, D., Yang, Q.: Transfer learning with graph co-regularization. In: Proc. of the 26th Conf. on Art. Int., pp. 1805–1818. AAAI (2012)Google Scholar
  12. 12.
    Muandet, K., Balduzzi, D., Schölkopf, B.: Domain generalization via invariant feature representation. In: Proc. of the 30th Int. Conf. on Mach. Learn., pp. 10–18 (2013)Google Scholar
  13. 13.
    Müller, K., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An introduction to kernel-based learning algorithms. IEEE Trans. on Neural Networks 12(2), 181–201 (2001)CrossRefGoogle Scholar
  14. 14.
    Pan, S.J., Tsang, I., Kwok, J., Yang, Q.: Domain adaptation via transfer component analysis. Trans. on Neural Networks 22(2), 199–210 (2011)CrossRefGoogle Scholar
  15. 15.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. on Know. and Data Eng. 22(10), 1345–1359 (2010)CrossRefGoogle Scholar
  16. 16.
    Pan, S.J., Zheng, V.W., Yang, Q., Hu, D.H.: Transfer learning for wifi-based indoor localization. In Proc. of the Workshop on Trans. Learn. for Complex Tasks, of the 23rd AAAI Conf. on Art. Int., pp. 43–48 (2008)Google Scholar
  17. 17.
    Persello, C., Bruzzone, L.: Relevant and invariant feature selection of hyperspectral images for domain generalization. In: Int. Geoscience and Remote Sensing Symposium (IGARSS), pp. 3562–3565. IEEE (2014)Google Scholar
  18. 18.
    Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D.: Dataset Shift in Machine Learning. The MIT Press (2009)Google Scholar
  19. 19.
    Schölkopf, B., Smola, A., Müller, K.: Kernel principal component analysis (1999)Google Scholar
  20. 20.
    Varnek, A., Gaudin, C., Marcou, G., Baskin, I., Pandey, A.K., Tetko, I.V.: Inductive transfer of knowledge: application of multi-task learning and feature net approaches to model tissue-air partition coefficients. J. of Chem. Inf. and Modeling 49(1), 133–144 (2009)CrossRefGoogle Scholar
  21. 21.
    Wang, C., Mahadevan, S.: Heterogeneous domain adaptation using manifold alignment. In: Proc. of The Twenty-Second Int. Joint Conf. on Art. Int., vol. 2, pp. 1541–1546. AAAI (2011)Google Scholar
  22. 22.
    Xue, Y., Liao, X., Carin, L., Krishnapuram, B.: Multitask learning for classication with Dirichlet process priors. J. Mach. Learn. Res. 35(8), 35–63 (2007)MathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Thomas Grubinger
    • 1
  • Adriana Birlutiu
    • 1
    • 2
  • Holger Schöner
    • 1
  • Thomas Natschläger
    • 1
  • Tom Heskes
    • 3
  1. 1.Data Analysis Systems, Software Competence Center HagenbergHagenbergAustria
  2. 2.Faculty of Science1 Decembrie 1918 University of Alba-IuliaAlba IuliaRomania
  3. 3.Institute for Computing and Information SciencesRadboud University NijmegenNijmegenNetherlands

Personalised recommendations