Regression Transfer Learning Based on Principal Curve
One of the basic ideas of present transfer learning algorithms is to find a common underlying representation among multiple related tasks as a bridge of transferring knowledge. Different with most transfer learning algorithms which are designed to solve classification problems, a new algorithm is proposed in this paper to solve multiple regression tasks. First, based on ”self-consistency” of principal curves, this algorithm utilizes non-parametric approach to find the principal curve passing through data sets of all tasks. We treat this curve as common-across-tasks representation. Second, the importance of every sample in target task is determined by computing the deviation from the principal curve and finally the weighted support vector regression is used to obtain a regression model. We simulate multiple related regression tasks using noisy Sinc data sets with various intensities and report experiments which demonstrate that the proposed algorithm can draw the useful information of multiple tasks and dramatically improve the performance relative to learning target task independently. Furthermore, we replace principal curve with support vector regression with model selection to find common representation and show the comparative results of these two algorithms.
KeywordsTransfer learning Support vector machine Principal curve
Unable to display preview. Download preview PDF.
- 2.Yin, J., Yang, Q., Ni, L.M.: Adaptive temporal radio maps for indoor location estimation. In: Proceedings of the 3rd IEEE International Conference on Pervasive Computing and Communications (PerCom 2005), pp. 85–94 (2005)Google Scholar
- 3.Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 99 (2009)Google Scholar
- 5.Shi, Y., Wang, J.: Similarity-based weighting approach for transfer learning. Technical report, Sun Yat-sen University, China (2009)Google Scholar
- 6.Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Proceedings of the 19th Annual Conference on Neural Information Processing Systems (NIPS), pp. 41–48. MIT Press, Cambridge (2007)Google Scholar
- 7.Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pp. 677–682. AAAI Press, Menlo Park (2008)Google Scholar
- 8.Hastie, T.: Principal curves and surfaces. Technical report, Department of Statistics, Stanford University (1983)Google Scholar
- 9.Hermann, T., Meinicke, P., Ritter, H.: Principal curve sonification. In: Proceedings of International Conference on Auditory Display, International Community for Auditory Displayn, pp. 81–86 (2000)Google Scholar
- 10.De’ath, G.: Principal curves: A new technique for indirect and direct gradient analysis. Ecology 80, 2237–2253 (1999)Google Scholar
- 11.Zhang, J., Wang, J.: An overview of principal curves. Chinese Journal of Computers 26, 129–146 (2003)Google Scholar
- 13.Qi, H., Wang, J.: A model for mining outliers from complex data sets. In: Proceedings of the 19th Annual ACM Symposium on Applied Computing, pp. 595–599. ACM, New York (2004)Google Scholar
- 14.Mao, W., Dong, L., Zhang, G.: Weighted solution path algorithm of support vector regression for abnormal data. In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4 (2008)Google Scholar