Multi-Task Learning Using Shared and Task Specific Information

  • P. K. Srijith
  • Shirish Shevade
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7665)


Multi-task learning solves multiple related learning problems simultaneously by sharing some common structure for improved generalization performance of each task. We propose a novel approach to multi-task learning which captures task similarity through a shared basis vector set. The variability across tasks is captured through task specific basis vector set. We use sparse support vector machine (SVM) algorithm to select the basis vector sets for the tasks. The approach results in a sparse model where the prediction is done using very few examples. The effectiveness of our approach is demonstrated through experiments on synthetic and real multi-task datasets.


Multi-task learning Support vector machines Kernel methods Sparse models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Caruana, R.: Multitask Learning. Machine Learning 28(1), 41–75 (1997)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Ando, R.K., Zhang, T.: A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data. J. Mach. Learn. Res. 6, 1817–1853 (2005)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Bakker, B., Heskes, T.: Task Clustering and Gating for Bayesian Multitask Learning. J. Mach. Learn. Res. 4, 83–99 (2003)Google Scholar
  4. 4.
    Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning Multiple Tasks with Kernel Methods. J. Mach. Learn. Res. 6, 615–637 (2005)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Xue, Y., Liao, X., Carin, L., Krishnapuram, B.: Multi-task Learning for Classification with Dirichlet Process Priors. J. Mach. Learn. Res. 8, 35–63 (2007)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Argyriou, A., Evgeniou, T., Pontil, M.: Convex Multi-task Feature Learning. Machine Learning 73(3), 243–272 (2008)CrossRefGoogle Scholar
  7. 7.
    Keerthi, S.S., Chapelle, O., DeCoste, D.: Building Support Vector Machines with Reduced Classifier Complexity. J. Mach. Learn. Res. 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Evgeniou, T., Pontil, M.: Regularized Multi-task Learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)Google Scholar
  9. 9.
    Liao, X., Carin, L.: Radial Basis Function Network for Multi-task Learning. In: Advances in Neural Information Processing Systems 18, pp. 795–802. MIT Press (2006)Google Scholar
  10. 10.
    Seeger, M.: Low Rank Updates for the Cholesky Decomposition. Technical report, University of California at Berkeley (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • P. K. Srijith
    • 1
  • Shirish Shevade
    • 1
  1. 1.Computer Science and AutomationIndian Institute of ScienceBangaloreIndia

Personalised recommendations