Leveraging Sequence Classification by Taxonomy-Based Multitask Learning

  • Christian Widmer
  • Jose Leiva
  • Yasemin Altun
  • Gunnar Rätsch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6044)


In this work we consider an inference task that biologists are very good at: deciphering biological processes by bringing together knowledge that has been obtained by experiments using various organisms, while respecting the differences and commonalities of these organisms. We look at this problem from an sequence analysis point of view, where we aim at solving the same classification task in different organisms. We investigate the challenge of combining information from several organisms, whereas we consider the relation between the organisms to be defined by a tree structure derived from their phylogeny. Multitask learning, a machine learning technique that recently received considerable attention, considers the problem of learning across tasks that are related to each other. We treat each organism as one task and present three novel multitask learning methods to handle situations in which the relationships among tasks can be described by a hierarchy. These algorithms are designed for large-scale applications and are therefore applicable to problems with a large number of training examples, which are frequently encountered in sequence analysis. We perform experimental analyses on synthetic data sets in order to illustrate the properties of our algorithms. Moreover, we consider a problem from genomic sequence analysis, namely splice site recognition, to illustrate the usefulness of our approach. We show that intelligently combining data from 15 eukaryotic organisms can indeed significantly improve the prediction performance compared to traditional learning approaches. On a broader perspective, we expect that algorithms like the ones presented in this work have the potential to complement and enrich the strategy of homology-based sequence analysis that are currently the quasi-standard in biological sequence analysis.


Support Vector Machine Baseline Method Hinge Loss Standard Support Vector Machine String Kernel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ando, R.K., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR 6, 1817–1853 (2005)MathSciNetGoogle Scholar
  2. 2.
    Ben-Hur, A., Ong, C.S., Sonnenburg, S., Schölkopf, B., Rätsch, G.: Support vector machines and kernels for computational biology. PLoS Comput. Biol. 4(10), e1000173 (2008)Google Scholar
  3. 3.
    Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Wortman, J.: Learning bounds for domain adaptation. In: Advances in Neural Information Processing Systems, vol. 20, pp. 129–136 (2008)Google Scholar
  4. 4.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001)Google Scholar
  5. 5.
    Daumé, H.: Frustratingly easy domain adaptation. In: ACL, The Association for Computer Linguistics (2007)Google Scholar
  6. 6.
    Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning multiple tasks with kernel methods. Journal of Machine Learning Research 6, 615–637 (2005)MathSciNetGoogle Scholar
  7. 7.
    Joachims, T.: SVMLight: Support Vector Machine. SVM-Light Support Vector Machine. University of Dortmund (1999),
  8. 8.
    Leslie, C., Eskin, E., Noble, W.S.: The spectrum kernel: A string kernel for SVM protein classification. In: Proceedings of the Pacific Symposium on Biocomputing, pp. 564–575 (2002)Google Scholar
  9. 9.
    Rätsch, G., Sonnenburg, S.: Accurate Splice Site Detection for Caenorhabditis elegans. MIT Press, Cambridge (2004)Google Scholar
  10. 10.
    Rätsch, G., Sonnenburg, S., Srinivasan, J., Witte, H., Müller, K.-R., Sommer, R., Schölkopf, B.: Improving the C. elegans genome annotation using machine learning. PLoS Computational Biology 3(2), e20 (2007)Google Scholar
  11. 11.
    Schweikert, G., Widmer, C., Schölkopf, B., Rätsch, G.: An empirical analysis of domain adaptation algorithms. In: Advances in Neural Information Processing System, NIPS, Vancouver, B.C., vol. 22 (2008)Google Scholar
  12. 12.
    Sonnenburg, S., Rätsch, G., Henschel, S., Widmer, C., Zien, A., de Bona, F., Gehl, C., Binder, A., Franc, V.: The shogun machine learning toolbox (under revision). Journal of Machine Learning Research (2010)Google Scholar
  13. 13.
    Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (1995)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Christian Widmer
    • 1
  • Jose Leiva
    • 1
    • 2
  • Yasemin Altun
    • 2
  • Gunnar Rätsch
    • 1
  1. 1.Friedrich Miescher LaboratoryMax Planck SocietyTübingenGermany
  2. 2.Max Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations