Abstract
Traditional machine learning works well under the assumption that the training data and test data are in the same distribution. However, in many real-world applications, this assumption does not hold. The research of knowledge transfer has received considerable interest recently in Natural Language Processing to improve the domain adaptation of machine learning. In this paper, we present a novel transfer learning framework called TPTSVM (Transfer Progressive Transductive Support Vector Machine), which combines transfer learning and semi-supervised learning. TPTSVM makes use of the limited labeled data in target domain to leverage a large amount of labeled data in source domain and queries the most confident instances in target domain. Experiments on two data sets show that TPTSVM algorithm always improves the classification performance compared to other state-of-the-art transfer learning approaches or semi-supervised approaches. Furthermore, our algorithm could be extended to multiple source domains easily.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Yin, X., Han, J., Yang, J., et al.: Efficient classification across multiple database relations: A crossmine approach. IEEE Transactions on Knowledge and Data Engineering 18(6), 770–783 (2006)
Kuncheva, L.I., Rodriguez, J.J.: Classifier ensembles with a random linear oracle. IEEE Transactions on Knowledge and Data Engineering 19(4), 500–508 (2007)
Baralis, E., Chiusano, S., Garza, P.: A lazy approach to associative classification. IEEE Transactions on Knowledge and Data Engineering 20(2), 156–171 (2008)
Fung, G.P.C., Yu, J.X., Lu, H., et al.: Text classification without negative examples revisit. IEEE Transactions on Knowledge and Data Engineering 18(1), 6–20 (2006)
Al-Mubaid, H., Umair, S.A.: A new text categorization technique using distributional clustering and learning logic. IEEE Transactions on Knowledge and Data Engineering 18(9), 1156–1165 (2006)
Sarinnapakorn, K., Kubat, M.: Combining subclassifiers in text categorization: A dst-based solution and a case study. IEEE Transactions on Knowledge and Data Engineering 19(12), 1638–1651 (2007)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345–1359 (2010)
Arnold, A., Nallapati, R., Cohen, W.W.: A comparative study of methods for transductive transfer learning. In: Seventh IEEE International Conference on ICDM Workshops 2007, pp. 77–82 (2007)
Thrun, S., Mitchell, T.M.: Learning one more thing. R. Carnegie-Mellon Univ. Pittsburgh Pa Dept. of Computer Science (1994)
Schmidhuber, J.: On learning how to learn learning strategies (1995)
Caruana, R.: Multitask learning. Springer, US (1998)
Blitzer, J., McDonald, R., Pereira, F.: Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 120–128. Association for Computational Linguistics (2006)
Daumé III, H., Marcu, D.: Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research 26(1), 101–126 (2006)
Pan, S.J., Zheng, V.W., Yang, Q., et al.: Transfer learning for wifi-based indoor localization. In: Association for the Advancement of Artificial Intelligence (AAAI) Workshop (2008)
Rosenstein, M.T., Marx, Z., Kaelbling, L.P., et al.: To transfer or not to transfer. In: NIPS 2005 Workshop on Transfer Learning, p. 898 (2005)
Dai, W., Yang, Q., Xue, G.R., et al.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 193–200. ACM (2007)
Eaton, E., des Jardins, M.: Selective transfer between learning tasks using task-based boosting. In: Twenty-Fifth AAAI Conference on Artificial Intelligence (2011)
Luo, C., Ji, Y., Dai, X., et al.: Active learning with transfer learning. In: Proceedings of ACL 2012 Student Research Workshop, pp. 13–18. Association for Computational Linguistics (2012)
Shao, M., Castillo, C., Gu, Z., et al.: Low-Rank Transfer Subspace Learning. In: Twelfth IEEE International Conference on ICDM Workshops 2012, pp. 1104–1109 (2012)
Negahban, S.N., Rubinstein, B.I.P., Gemmell, J.G.: Scaling multiple-source entity resolution using statistically efficient transfer learning. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 2224–2228. ACM (2012)
Ju, S., Li, S., Su, Y., et al.: Dual word and document seed selection for semi-supervised sentiment classification. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 2295–2298. ACM (2012)
Joachims, T.: Transductive inference for text classification using support vector machines. In: Machine Learning-International Workshop Then Conference, pp. 200–209. Morgan Kaufmann Publishers, Inc. (2009)
Chen, Y., Wang, G., Dong, S.: Learning with progressive transductive support vector machine. Pattern Recognition Letters 24(12), 1845–1855 (2003)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
Joachims, T.: Learning to classify text using support vector machines: Methods, theory and algorithms. Kluwer Academic Publishers (2002)
Daumé III, H., Kumar, A., Saha, A.: Frustratingly easy semi-supervised domain adaptation. In: Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, pp. 53–59. Association for Computational Linguistics (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhou, H., Zhang, Y., Huang, D., Li, L. (2013). Semi-supervised Learning with Transfer Learning. In: Sun, M., Zhang, M., Lin, D., Wang, H. (eds) Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. NLP-NABD CCL 2013 2013. Lecture Notes in Computer Science(), vol 8202. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41491-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-41491-6_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41490-9
Online ISBN: 978-3-642-41491-6
eBook Packages: Computer ScienceComputer Science (R0)