Regression Transfer Learning Based on Principal Curve

  • Wentao Mao
  • Guirong Yan
  • Junqing Bai
  • Hao Li
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6063)


One of the basic ideas of present transfer learning algorithms is to find a common underlying representation among multiple related tasks as a bridge of transferring knowledge. Different with most transfer learning algorithms which are designed to solve classification problems, a new algorithm is proposed in this paper to solve multiple regression tasks. First, based on ”self-consistency” of principal curves, this algorithm utilizes non-parametric approach to find the principal curve passing through data sets of all tasks. We treat this curve as common-across-tasks representation. Second, the importance of every sample in target task is determined by computing the deviation from the principal curve and finally the weighted support vector regression is used to obtain a regression model. We simulate multiple related regression tasks using noisy Sinc data sets with various intensities and report experiments which demonstrate that the proposed algorithm can draw the useful information of multiple tasks and dramatically improve the performance relative to learning target task independently. Furthermore, we replace principal curve with support vector regression with model selection to find common representation and show the comparative results of these two algorithms.


Transfer learning Support vector machine Principal curve 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dai, W., Xue, G., Yang, Q., Yu, Y.: Co-clustering based classification for out-of-domain documents. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 210–219. ACM, New York (2007)CrossRefGoogle Scholar
  2. 2.
    Yin, J., Yang, Q., Ni, L.M.: Adaptive temporal radio maps for indoor location estimation. In: Proceedings of the 3rd IEEE International Conference on Pervasive Computing and Communications (PerCom 2005), pp. 85–94 (2005)Google Scholar
  3. 3.
    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 99 (2009)Google Scholar
  4. 4.
    Dai, W., Yang, Q., Xue, G., Yu, Y.: Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning (ICML), pp. 193–200. ACM, New York (2007)CrossRefGoogle Scholar
  5. 5.
    Shi, Y., Wang, J.: Similarity-based weighting approach for transfer learning. Technical report, Sun Yat-sen University, China (2009)Google Scholar
  6. 6.
    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Proceedings of the 19th Annual Conference on Neural Information Processing Systems (NIPS), pp. 41–48. MIT Press, Cambridge (2007)Google Scholar
  7. 7.
    Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pp. 677–682. AAAI Press, Menlo Park (2008)Google Scholar
  8. 8.
    Hastie, T.: Principal curves and surfaces. Technical report, Department of Statistics, Stanford University (1983)Google Scholar
  9. 9.
    Hermann, T., Meinicke, P., Ritter, H.: Principal curve sonification. In: Proceedings of International Conference on Auditory Display, International Community for Auditory Displayn, pp. 81–86 (2000)Google Scholar
  10. 10.
    De’ath, G.: Principal curves: A new technique for indirect and direct gradient analysis. Ecology 80, 2237–2253 (1999)Google Scholar
  11. 11.
    Zhang, J., Wang, J.: An overview of principal curves. Chinese Journal of Computers 26, 129–146 (2003)Google Scholar
  12. 12.
    Kégl, B., Krzyzak, A., Linder, T., Zeger, K.: Learning and design of principal curves. IEEE Transactions on Pattern Recognition and Machine Intelligence 22, 281–297 (2000)CrossRefGoogle Scholar
  13. 13.
    Qi, H., Wang, J.: A model for mining outliers from complex data sets. In: Proceedings of the 19th Annual ACM Symposium on Applied Computing, pp. 595–599. ACM, New York (2004)Google Scholar
  14. 14.
    Mao, W., Dong, L., Zhang, G.: Weighted solution path algorithm of support vector regression for abnormal data. In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4 (2008)Google Scholar
  15. 15.
    Tay, F., Cao, L.: Modified support vector machines in financial time series forecasting. Neurocomputing 48, 847–862 (2003)CrossRefGoogle Scholar
  16. 16.
    Wang, G., Yeung, D.Y., Lochovsky, F.H.: A new solution path algorithm in support vector regression. IEEE Transactions on Neural Networks 19(10), 1753–1767 (2008)CrossRefGoogle Scholar
  17. 17.
    Hu, D., Mao, W., Zhao, J., Yan, G.: Application of lssvm-pso to load identification in frequency domain. In: Proceedings of Artificial Intelligence and Computational Intelligence, pp. 231–240. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Wentao Mao
    • 1
  • Guirong Yan
    • 1
  • Junqing Bai
    • 1
  • Hao Li
    • 1
  1. 1.The Key Laboratory of Strength and Vibration of Ministry of EducationXi’an Jiaotong UniversityXi’anChina

Personalised recommendations