Skip to main content
SpringerLink
Log in

Novel Inductive and Transductive Transfer Learning Approaches Based on Support Vector Learning

  • Chapter
  • First Online:
Support Vector Machines Applications

Abstract

In this chapter, two novel transfer learning approaches based on support vector learning are involved. For inductive transfer learning, the knowledge-leverage-based TSK fuzzy system (KL-TSK-FS) is proposed, which demonstrates the good privacy-protection abilities and strong adaptability for the situations where the data are only partially available from the target domain while some useful knowledge of the source domains is available. For transductive transfer learning, domain adaptation kernelized support vector machine (DAKSVM) and its two extensions are proposed, which can reduce the distribution gap between different domains in an RKHS as much as possible by integrating the large margin learner with the proposed generalized projected maximum distribution distance (GPMDD) metric.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)

    Article  Google Scholar 

  2. Deng, Z.H., Choi, K.S., Chung, F.L., et~al.: Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation. IEEE Trans. Fuzzy Syst. 19(2), 210–226 (2011)

    Google Scholar 

  3. Liao, X., Xue, Y., Carin, L.: Logistic regression with an auxiliary data source. In: Proceedings of 21st International Conference Machine Learning, pp. 505–512 (2005)

    Google Scholar 

  4. Huang, J., Smola, A., Gretton, A., et~al.: Correcting sample selection bias by unlabeled data. In: Proceedings of 19th Annual Conference Neural Information Processing Systems, pp. 601–608 (2007)

    Google Scholar 

  5. Bickel, S., Brückner, M., Scheffer, T.: Discriminative learning for differing training and test distributions. In: Proceedings of 24th International Conference Machine Learning, pp. 81–88 (2007)

    Google Scholar 

  6. Sugiyama, M., Nakajima, S., Kashima, H., et~al.: Direct importance estimation with model selection and its application to covariate shift adaptation. In: Proceedings of 20th Annual Conference Neural Information Processing Systems, Bangkok (December 2008)

    Google Scholar 

  7. Lawrence, N.D., Platt, J.C.: Learning to learn with the informative vector machine. In: Proceedings of 21st International Conference Machine Learning (July 2004)

    Google Scholar 

  8. Schwaighofer, A., Tresp, V., Yu, K.: Learning Gaussian process kernels via hierarchical Bayes. In: Proceedings 17th Annual Conference Neural Information Processing Systems, pp. 1209–1216 (2005)

    Google Scholar 

  9. Gao, J., Fan, W., Jiang, J., et~al.: Knowledge transfer via multiple model local structure mapping. In: Proceedings of 14th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 283–291 (August 2008)

    Google Scholar 

  10. Mihalkova, L., Huynh, T., Mooney, R.J.: Mapping and revising Markov logic networks for transfer learning. In: Proceedings of 22nd Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence, pp. 608–614 (July 2007)

    Google Scholar 

  11. Mihalkova, L., Mooney, R.J.: Transfer learning by mapping with minimal target data. In: Proceedings of Association for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks (July 2008)

    Google Scholar 

  12. Davis, J., Domingos, P.: Deep transfer via second-order Markov logic. In: Proceedings of Association for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks (July 2008)

    Google Scholar 

  13. Pan, S.J., Tsang, I.W., Kwok, J.T., et~al.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)

    Google Scholar 

  14. Duan, L.X., Xu, D., Tsang, I.W.: Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans. Neural Netw. Learn. Syst. 23(3), 504–518 (2012)

    Article  Google Scholar 

  15. Duan, L.X., Tsang, I.W., Xu, D.: Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 465–479 (2012)

    Article  Google Scholar 

  16. Dai, W., Yang, Q., Xue, G., et~al.: Self-taught clustering. In: Proceedings of 25th International Conference Machine Learning, pp. 200–207 (July 2008)

    Google Scholar 

  17. Jiang, W.H., Chung, F.L.: Transfer spectral clustering. In: Proceedings of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), Bristol (24–28 September 2012)

    Google Scholar 

  18. Wang, Z., Song, Y., Zhang, C.: Transferred dimensionality reduction. In: Proceedings of European Conference Machine Learning and Knowledge Discovery in Databases (ECML/PKDD’08), pp. 550–565 (September 2008)

    Google Scholar 

  19. Gong, B., Shi, Y., Sha, F., et~al.: Geodesic flow kernel for unsupervised domain adaptation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2066–2073 (2012)

    Google Scholar 

  20. Yang, P., Tan, Q., Ding, Y.: Bayesian task-level transfer learning for non-linear regression. In: Proceedings of International Conference on Computer Science and Software Engineering, pp. 62–65 (2008)

    Google Scholar 

  21. Borzemski, L., Starczewski, G.: Application of transfer regression to TCP throughput prediction. In: Proceedings of First Asian Conference on Intelligent Information and Database Systems, pp. 28–33 (2009)

    Google Scholar 

  22. Mao, W., Yan, G., Bai, J., et~al.: Regression transfer learning based on principal curve. Lect. Notes Comput. Sci. 6063, 365–372 (2010)

    Google Scholar 

  23. Liu, J., Chen, Y., Zhang, Y.: Transfer regression model for indoor 3D location estimation. Lect. Notes Comput. Sci. 5916, 603–613 (2010)

    Article  Google Scholar 

  24. Pardoe, D., Stone, P.: Boosting for regression transfer. In: Proceedings of International Conference on Machine Learning, pp. 863–870 (2010)

    Google Scholar 

  25. Dai, W., Yang, Q., Xue, G., et~al.: Boosting for transfer learning. In: Proceedings of 24th International Conference on Machine Learning, pp. 193–200 (June 2007)

    Google Scholar 

  26. Wu, P., Dietterich, T.G.: Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of 21st International Conference on Machine Learning (July 2004)

    Google Scholar 

  27. Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117 (August 2004)

    Google Scholar 

  28. Qduanz, B., Huan, J.: Large margin transductive transfer learning. In: Proceedings of 18th ACM conference on Information and knowledge management (CIKM), pp. 1327–1336. ACM, New York (2009)

    Google Scholar 

  29. Duan, L., Tsang, I.W., Xu, D., et~al.: Domain transfer SVM for video concept detection. In: Proceedings IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1375–1381 (2009)

    Google Scholar 

  30. Bruzzone, L., Marconcini, M.: Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 770–787 (2010)

    Article  Google Scholar 

  31. Ben-David, S., Blitzer, J., Crammer, K., et~al.: Analysis of representations for domain adaptation. In: NIPS (2007)

    Google Scholar 

  32. Gretton, A., Harchaoui, Z., Fukumizu, K., Sriperumbudur, B.: A fast, consistent Kernel two-sample test. In: Advances in Neural Information Processing Systems, vol. 22, pp. 673–681. MIT, Cambridge (2010)

    Google Scholar 

  33. Leski, J.: TSK-fuzzy modeling based on ε-insensitive learning. IEEE Trans. Fuzzy Syst. 13(2), 181–193 (2005)

    Article  Google Scholar 

  34. Wang, L.X.: Adaptive fuzzy systems and control: design and stability analysis. Prentice-Hall, Upper Saddle River (1994)

    Google Scholar 

  35. Jang, J.S.R., Sun, C.T., Mizutani, E.: Neuro-fuzzy and soft-computing. Prentice-Hall, Upper Saddle River (1997)

    Google Scholar 

  36. Chen, B., Lam, W., Tsang, I.W., et~al.: Location and scatter matching for dataset shift in text mining. In: Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM 2010), Sydney (December 2010)

    Google Scholar 

  37. Huang, J., Smola, A., Gretton, A., et~al.: Correcting sample selection bias by unlabeled data. In: Proceedings of Twentieth Annual Conference on Neural Information Processing Systems (2006)

    Google Scholar 

  38. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its application to modeling and control. IEEE Trans. Syst. Man Cybern. 15(1), 116–132 (1985)

    Article  MATH  Google Scholar 

  39. Mamdani, E.H.: Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans. Comput. C-26(12), 1182–1191 (1977)

    Article  Google Scholar 

  40. Azeem, M.F., Hanmandlu, M., Ahmad, N.: Generalization of adaptive neural-fuzzy inference systems. IEEE Trans. Neural Netw. 11(6), 1332–1346 (2000)

    Article  Google Scholar 

  41. Deng, Z.H., Choi, K.S., Chung, F.L., et~al.: Enhanced soft subspace clustering integrating within-cluster and between-cluster information. Pattern Recognit. 43(3), 767–781 (2010)

    Google Scholar 

  42. Jang, J.S.R.: ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man Cybern. 23(3), 665–685 (1993)

    Article  Google Scholar 

  43. Tsang, I.W., Kwok, J.T., Zurada, J.M.: Generalized core vector machines. IEEE Trans. Neural Netw. 17(5), 1126–1140 (2006)

    Article  Google Scholar 

  44. Deng, Z.H., Jiang, Y.Z., Chung, F.L., et~al.: Knowledge-leverage based fuzzy system and its modeling. IEEE Trans. Fuzzy Syst. (2012). doi:10.1109/TFUZZ.2012.2212444

  45. Fan, R.E., Chen, P.H., Lin, C.J.: Working Set selection using second order information for training support vector machines. J. Mach. Learn. Res. 6, 1889–1918 (2005)

    MATH  MathSciNet  Google Scholar 

  46. Hofmann, T., Schölkopf, B., Smola, A.J.: Kernel methods in machine learning. Ann. Stat. 36, 1171–1220 (2007)

    Article  Google Scholar 

  47. Sriperumbadur, B.K., Fukumizu, K., Gretton, A., et~al.: Kernel choice and classifiability for RKHS embeddings of probability distributions. In: Advances in Neural Information Processing Systems, vol. 22, pp. 1750–1758. MIT, Cambridge (2010)

    Google Scholar 

  48. Smola, A.J., Gretton, A., Song, L., et~al.: A Hilbert space embedding for distributions. In: Proceedings of 18th International Conference on Algorithmic Learn. Theory, Sendai, pp. 13–31 (October 2007)

    Google Scholar 

  49. Gretton, A., Harchaoui, Z., Fukumizu, K., et~al.: A fast, consistent kernel two-sample test. In: Advances in Neural Information Processing Systems, vol. 22, pp. 673–681. MIT, Cambridge (2010)

    Google Scholar 

  50. Borgwardt, K.M., Gretton, A., Rasch, M.J., et~al.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics (ISMB) 22(14), e49–e57 (2006)

    Google Scholar 

  51. Sriperumbudur, B.K., Gretton, A., Fukumizu, K., et~al.: Hilbert space embeddings and metrics on probability measures. J. Mach. Learn. Res. 11(3), 1517–1561 (2010)

    Google Scholar 

  52. Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  MathSciNet  Google Scholar 

  53. Wu, Y., Liu, Y.: Robust truncated hinge loss support vector machines. J. Am. Stat. Assoc. 102(479), 974–983 (2007)

    Article  MATH  Google Scholar 

  54. Scholkopf, B., Herbrich, R., Smola, A.J.: Generalized representer theorem. In: Proceedings of COLT’2001, pp. 416–426. Springer, Amsterdam (2001)

    Google Scholar 

  55. Szedmak, S., Shawe-Taylor, J.: Multiclass learning at one-class complexity. Technical Report No. 1508, School of Electronics and Computer Science, Southampton (2005)

    Google Scholar 

  56. Schölkopf, B., Smola, A.J., Williamson, R., et~al.: New support vector algorithms. Neural Comput. 12(5), 1207–1245 (2000)

    Google Scholar 

  57. Chang, C.C., Lin, C.J.: Training v-support vector classifiers: theory and algorithms. Neural Comput. 13(9), 2119–2147 (2001)

    Article  MATH  Google Scholar 

  58. Sindhwani, V., Belkin, M., Niyogi, P.: The geometric basis of semi-supervised learning. In: Chapelle, O., Schölkopf, B., Zien, A. (eds.) Semi-Supervised Learning. MIT, Cambridge (2006)

    Google Scholar 

  59. Tsang, I.W., Kwok, J.T.: Very large scale manifold regularization using core vector machines. In: NIPS 2005 Workshop on Large Scale Kernel Machines (2005)

    Google Scholar 

  60. Juang, C.F., Chiu, S.H., Shiu, S.J.: Fuzzy system learned through fuzzy clustering and support vector machine for human skin color segmentation. IEEE Trans. Syst. Man Cybern. 37(6), 1077–1087 (2007)

    Article  Google Scholar 

  61. Juang, C.F., Hsieh, C.D.: TS-fuzzy system-based support vector regression. Fuzzy Sets Syst. 60(17), 2486–2504 (2009)

    Article  MathSciNet  Google Scholar 

  62. Abril, L.G., Angulo, C., Velasco, F., et~al.: A note on the bias in SVMs for multi classification. IEEE Trans. Neural Netw. 19(4), 723–725 (2008)

    Google Scholar 

  63. Ling, X., Dai, W., Xue, G., et~al.: Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York (2008)

    Google Scholar 

  64. Xiang, E.W., Cao, B., Hu, D.H., et~al.: Bridging domains using world wide knowledge for transfer learning. IEEE Trans. Knowl. Data Eng. 22(6), 770–783 (2010)

    Google Scholar 

  65. Dai, W., Xue, G.R., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, pp. 210–219. ACM, New York (August 2007)

    Google Scholar 

  66. Bickel, S.: ECML-PKDD discovery challenge 2006 overview. In: Proceedings of ECML/PKDD on Discovery Challenge Workshop (2006)

    Google Scholar 

  67. Phan, X.H., Nguyen, M.L., Horiguchi, S.: Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: Proceedings of 17th International World Wide Web Conference (WWW’08), pp. 91–100 (April 2008)

    Google Scholar 

  68. Beitzel, S.M., Jensen, E.C., Frieder, O., et~al.: Improving automatic query classification via semi-supervised learning. In: Proceedings of Fifth IEEE International Conference Data Mining (ICDM’05), pp. 42–49 (November 2005)

    Google Scholar 

  69. Blitzer, J., Dredze, M., Pereira, F., et~al.: Boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of 45th Annual Meeting of the Association for Computational Linguistics (ACL’07), pp. 440–447 (June 2007)

    Google Scholar 

  70. Alpaydin, E.: Introduction to machine learning. MIT, Cambridge (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Digital Media, Jiangnan University, Wuxi, Jiangsu, P.R. China

    Zhaohong Deng & Shitong Wang

Authors
  1. Zhaohong Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shitong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shitong Wang .

Editor information

Editors and Affiliations

  1. Honeywell, Golden Valley, Minnesota, USA

    Yunqian Ma

  2. West Virginia University, Morgantown, West Virginia, USA

    Guodong Guo

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Deng, Z., Wang, S. (2014). Novel Inductive and Transductive Transfer Learning Approaches Based on Support Vector Learning. In: Ma, Y., Guo, G. (eds) Support Vector Machines Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-02300-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02300-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02299-4

  • Online ISBN: 978-3-319-02300-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation