Skip to main content
SpringerLink
Log in

Inductive transfer learning for unlabeled target-domain via hybrid regularization

  • Articles/Computer Science & Technology
  • Published:
Chinese Science Bulletin

Abstract

Recent years have witnessed an increasing interest in transfer learning. This paper deals with the classification problem that the target-domain with a different distribution from the source-domain is totally unlabeled, and aims to build an inductive model for unseen data. Firstly, we analyze the problem of class ratio drift in the previous work of transductive transfer learning, and propose to use a normalization method to move towards the desired class ratio. Furthermore, we develop a hybrid regularization framework for inductive transfer learning. It considers three factors, including the distribution geometry of the target-domain by manifold regularization, the entropy value of prediction probability by entropy regularization, and the class prior by expectation regularization. This framework is used to adapt the inductive model learnt from the source-domain to the target-domain. Finally, the experiments on the real-world text data show the effectiveness of our inductive method of transfer learning. Meanwhile, it can handle unseen test points.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Dai W Y, Yang Q, Xue G R, et al. Boosting for Transfer Learning. In: Ghahramani Z B, eds. Proceeding of 24th International Conference on Machine Learning, 2007 Jun 20–24, Corvalis, Oregon. ACM, 2007. 193–200

    Chapter  Google Scholar 

  2. Raina R, Battle A, Lee H, et al. Self-taught Learning: Transfer Learning from Unlabeled Data. In: Ghahramani Z B, eds. Proceeding of 24th International Conference on Machine Learning, 2007 Jun 20–24, Corvalis, Oregon. ACM, 2007. 759–766

    Chapter  Google Scholar 

  3. Dai W Y, Xue G R, Yang Q, et al. Co-clustering based Classification for Out-of-domain Documents. In: Berkhin P, Caruana R, Wu X D, et al, eds. Proceeding of 13th ACM International Conference on Knowledge Discovery and Data Mining, 2007, Aug 12–15, San Jose, California. ACM, 2007. 210–219

    Chapter  Google Scholar 

  4. Xing D K, Dai W Y, Xue G R, et al. Bridged Refinement for Transfer Learning. In: Joost N K, Jacek K, Ramon L, et al, eds. Proceeding of 11th European Conference on Practice of Knowledge Discovery in Databases, 2007 Sep 17–21, Warsaw Poland. Springer, 2007. 324–335

    Google Scholar 

  5. Liao X J, Xue Y, Carin L, et al. Logistic Regression with an Auxiliary Data Source. In: Raedt L D, Wrobel S, eds. Proceeding of 22th International Conference on Machine Learning, 2007 Aug 7–11, Bonn, Germany. ACM, 2005. 505–512

    Google Scholar 

  6. Yang J, Yan R, Hauptmann A G, et al. Cross-domain Video Concept Detection Using Adaptive SVMs. In Rainer L, Anand R P, Alan H, et al, eds. Proceeding of 15th International Conference on Multimedia, 2007 Sep 24–29, Augsburg Germany. ACM, 2007. 188–197

    Chapter  Google Scholar 

  7. Wu P C, Dietterich T G, et al. Improving SVM Accuracy by Training on Auxiliary Data Sources. In: Brodley C E, eds. Proceeding of 21th International Conference on Machine Learning, 2004 Jul 4–8, Banff, Alberta, Canada. ACM, 2004. 871–878

    Google Scholar 

  8. Mahmud M M H, Ray S, et al. Transfer Learning Using Kolmogorov Complexity: Basic Theory and Empirical Evaluations. Technical Report, UIUC-DCS-R-2007-2875, Department of Computer Science, University of Illinois at Urbana-Champaign. 2007

  9. Ben-David S, Blitzer J, Crammer K, Pereira F, et al. Analysis of Representations for Domain Adaptation. In: Koller D, Singer Y, Platt J, et al, eds. Proceeding of Advances in Neural Information Processing Systems 20, MIT Press, Cambridge, MA, 2007(20): 137–144

    Google Scholar 

  10. Dai W Y, Xue G R, Yang Q, et al. Transferring Naive Bayes Classifiers for Text Classification. In: John C, Peyman F, Simon P, et al, eds. Proceeding of 22nd Conference on Artificial Intelligence, 2007 Jul 22–26, Vancouver, British Columbia. AAAI Press, 2007. 540–545

    Google Scholar 

  11. Samarth S, Sylvian R, et al. Cross Domain Knowledge Transfer Using Structured Representations. In: Proceeding of 21nd Conference on Artificial Intelligence, 2006 Jul 16–22, Boston, Massachusetts. AAAI Press, 2006

    Google Scholar 

  12. Ling X, Dai W Y, Xue G R, et al. Spectral Domain-Transfer Learning. In: Li Y, Liu B, Sunita S, et al, eds. Proceeding of 14th ACM International Conference on Knowledge Discovery and Data Mining, 2008, Aug 24–27, Las Vegas, Nevada. ACM, 2008. 488–496

    Chapter  Google Scholar 

  13. Mahmud, M M H. On Universal Transfer Learning. In: Rocco M H, Servedio R A, Takimoto E, et al, eds. Proceeding of 18th International Conference on Algorithmic Learning Theory, 2007, Oct 1–4, Sendai, Japan. LNCS, 2007. 135–149

    Chapter  Google Scholar 

  14. Belkin M, Niyogi P, Sindhwani V, et al. Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research, 2006: 2399–2434

  15. Grandvalet Y, Bengio Y, et al. Semi-supervised Learning by Entropy Minimization. Proceeding of 19th Conference on Neural Information Processing Systems, 2005 Dec 5–8, Vancouver, British Columbia. MIT Press, 2005. 529–536

    Google Scholar 

  16. Mann G S, McCallum A, et al. Simple, Robust, Scalable Semi-supervised Learning via Expectation Regularization. In: Ghahramani Z B, eds. Proceeding of 24th International Conference on Machine Learning, 2007 Jun 20–24, Corvalis, Oregon. ACM, 2007. 593–600

    Chapter  Google Scholar 

  17. Wang F, Zhang C S, et al. Label Propagation through Linear Neighborhoods. IEEE Transations on Knowledge and Data Engineering, 2008: 55–67

  18. Joachims T. Making Large-scale SVM Learning Practical. In: Scholkopf B, Burges C, Smola A, et al, eds. Proceeding of Advances in Kernel Methods, 1999. MIT Press, Cambridge, 1999. 169–184

    Google Scholar 

  19. Davie H, Stanley L, et al. Applied Logistic Regression. Wiley, New York, 2000

    Google Scholar 

  20. Joachims T. Transductive Inference for Text Classification Using Support Vector Machines. In: Bratko I, Dzeroski S, eds. Proceeding of 16th International Conference on Machine Learning, 1999 Jun 27–30, Bled, Slovenia. ACM, 1999. 200–209

    Google Scholar 

  21. Joachims T. Transductive Learning via Spectral Graph Partitioning. In: Tom F, Nina M, eds. Proceeding of 20th International Conference on Machine Learning, 2003 Aug 21–24, Washington DC. ACM, 2003. 290–297

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    FuZhen Zhuang, Qing He & ZhongZhi Shi

  2. Hewlett Packard Labs China, Beijing, 100084, China

    Ping Luo

  3. Graduate University of Chinese Academy of Sciences, Beijing, 100190, China

    FuZhen Zhuang

Authors
  1. FuZhen Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ping Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qing He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. ZhongZhi Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to FuZhen Zhuang.

Additional information

Supported by the National Science Foundation of China (Grant Nos. 60435010, 60675010), National High Technology Research and Development of China (Grant Nos. 2006AA01Z128, 2007AA01Z132), National Basic Research Priorities Programme (Grant No. 2007CB311004) and National Science and Technology Support Plan (Grant No. 2006BAC08B06)

Electronic supplementary material

About this article

Cite this article

Zhuang, F., Luo, P., He, Q. et al. Inductive transfer learning for unlabeled target-domain via hybrid regularization. Chin. Sci. Bull. 54, 2470–2478 (2009). https://doi.org/10.1007/s11434-009-0171-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11434-009-0171-x

Keywords

Search

Navigation