Regularization-Based Multitask Learning With Applications to Genome Biology and Biological Imaging

KI - Künstliche Intelligenz volume 28, pages29–33(2014)Cite this article

Abstract

The aim of multitask learning is to improve the generalization performance of a set of related tasks by exploiting complementary information about the tasks. In this paper, we review established approaches for regularization based multitask learning, sketch some recent developments, and demonstrate their applications in Computational Biology and Biological Imaging.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Blanchard G, Lee G, Scott C (2011) Generalizing from several related classification tasks to a new unlabeled sample. In: Shawe-Taylor J, Zemel RS, Bartlett P, Pereira FCN, Weinberger KQ (eds) Advances in neural information processing systems, vol 24. MIT Press, Cambridge, pp 2178–2186

  2. 2.

    Bonilla E, Chai KM, Williams C (2008) Multi-task Gaussian process prediction. Adv Neural Inform Process Syst 20:153–160

    Google Scholar 

  3. 3.

    Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on computational learning theory, COLT ’92. ACM, New York, pp 144–152

  4. 4.

    Caruana R (1993) Multitask learning: a knowledge-based source of inductive bias. In: ICML. Morgan Kaufmann, San Francisco, pp 41–48

  5. 5.

    Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  6. 6.

    Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  7. 7.

    Daumé H (2007) Frustratingly easy domain adaptation. In: Annual meeting-association for computational linguistics, vol 45, p 256

  8. 8.

    Evgeniou T, Micchelli CA, Pontil M (2005) Learning multiple tasks with kernel methods. J Mach Learn Res 6(1):615–637

    MATH  MathSciNet  Google Scholar 

  9. 9.

    Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: International conference on knowledge discovery and data mining, p 109

  10. 10.

    Jacob L, Vert J (2008) Efficient peptide-MHC-I binding prediction for alleles with few known binders. Bioinformatics (Oxford, England) 24(3):358–66

    Article  Google Scholar 

  11. 11.

    Lou X, Hamprecht FA (2011) Structured learning for cell tracking. In: Advances in neural information processing systems, vol 24 (NIPS 2011)

  12. 12.

    Lou X, Hamprecht FA (2012) Structured learning from partial annotations. In: The 29th international conference on machine learning (ICML 2012)

  13. 13.

    Lou X, Koethe U, Wittbrodt J, Hamprecht FA (2012) Learning to segment dense cell nuclei with shape prior. In: IEEE conference on computer vision and pattern recognition (CVPR), 2012, pp 1012–1018. IEEE

  14. 14.

    Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Google Scholar 

  15. 15.

    Schweikert G, Widmer C, Schölkopf B, Rätsch G (2009) An empirical analysis of domain adaptation algorithms for genomic sequence analysis. In: Koller D, Schuurmans D, Bengio Y, Bottou L (eds) Advances in neural information processing systems, NIPS, vol 21, pp 1433–1440

  16. 16.

    Tsochantaridis I, Joachims T, Hofmann T, Altun Y, Singer Y (2006) Large margin methods for structured and interdependent output variables. J Mach Learn Res 6(2):1453

    Google Scholar 

  17. 17.

    Vapnik VN, Chervonenkis AYa (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16(2):264–280

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Widmer C, Leiva J, Altun Y, Rätsch G (2010) Leveraging sequence classification by taxonomy-based multitask learning. In: Berger B (ed) Research in computational molecular biology. Springer, Berlin, pp 522–534

  19. 19.

    Widmer C, Rätsch G (2012) Multitask learning in computational biology. In: JMLR W&CP. ICML 2011 Unsupervised and transfer learning workshop, vol 27, pp 207–216

  20. 20.

    Widmer C, Toussaint NC, Altun Y, Rätsch G (2010) Inferring latent task structure for multitask learning by multiple kernel learning. BMC Bioinform 11 Suppl 8(Suppl 8):S5

    Google Scholar 

  21. 21.

    Zhang Y, Yeung DY (2010) A convex formulation for learning task relationships in multi-task learning. arXiv preprint arXiv:1203.3536

Download references

Acknowledgments

We thank Klaus-Robert Müller and Mehryar Mohri for inspiring and helpful discussions. This work was supported by the German Research Foundation (DFG) under MU 987/6-1 and RA 1894/1-1 as well as by the European Community’s 7th Framework Programme under the PASCAL2 Network of Excellence (ICT-216886). Marius Kloft acknowledges a postdoctoral fellowship by the German Research Foundation (DFG). We also thank Anna-Katerina Hadjantonakis and Minjung Kang for the 3D image data.

Author information

Affiliations

  1. Computational Biology Center, Memorial Sloan-Kettering Cancer Center, 415 E 68th Street, New York, NY, 10065, USA

    Christian Widmer, Marius Kloft, Xinghua Lou & Gunnar Rätsch

  2. Machine Learning Group, Technische Universität Berlin, Marchstr. 23, 10587, Berlin, Germany

    Christian Widmer

  3. Courant Institute of Mathematical Sciences, 251 Mercer St, New York, NY, 10012, USA

    Marius Kloft

Authors
  1. Christian Widmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marius Kloft
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinghua Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gunnar Rätsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gunnar Rätsch.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Widmer, C., Kloft, M., Lou, X. et al. Regularization-Based Multitask Learning With Applications to Genome Biology and Biological Imaging. Künstl Intell 28, 29–33 (2014). https://doi.org/10.1007/s13218-013-0283-y

Download citation

Keywords

  • Multitask learning
  • Transfer learning
  • Regularized risk minimization