Semi-supervised Abstraction-Augmented String Kernel for Multi-level Bio-Relation Extraction

  • Pavel Kuksa
  • Yanjun Qi
  • Bing Bai
  • Ronan Collobert
  • Jason Weston
  • Vladimir Pavlovic
  • Xia Ning
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6322)


Bio-relation extraction (bRE), an important goal in bio-text mining, involves subtasks identifying relationships between bio-entities in text at multiple levels, e.g., at the article, sentence or relation level. A key limitation of current bRE systems is that they are restricted by the availability of annotated corpora. In this work we introduce a semi-supervised approach that can tackle multi-level bRE via string comparisons with mismatches in the string kernel framework. Our string kernel implements an abstraction step, which groups similar words to generate more abstract entities, which can be learnt with unlabeled data. Specifically, two unsupervised models are proposed to capture contextual (local or global) semantic similarities between words from a large unannotated corpus. This Abstraction-augmented String Kernel (ASK) allows for better generalization of patterns learned from annotated data and provides a unified framework for solving bRE with multiple degrees of detail. ASK shows effective improvements over classic string kernels on four datasets and achieves state-of-the-art bRE performance without the need for complex linguistic features.


Semi-supervised string kernel Relation extraction Sequence classification Learning with auxiliary information 


  1. 1.
    Airola, A., Pyysalo, S., Bjorne, J., Pahikkala, T., Ginter, F., Salakoski, T.: All-paths graph kernel for protein-protein interaction extraction with evaluation of cross-corpus learning. BMC Bioinformatics 9(S11), S2 (2008)Google Scholar
  2. 2.
    Ando, R.K., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. J. of Machine Learning Research 6, 1817–1853 (2005)MathSciNetGoogle Scholar
  3. 3.
    Bunescu, R., Mooney, R.: Subsequence kernels for relation extraction. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) NIPS 2006, pp. 171–178 (2006)Google Scholar
  4. 4.
    Cancedda, N., Gaussier, E., Goutte, C., Renders, J.M.: Word sequence kernels. J. Mach. Learn. Res. 3, 1059–1082 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning (Adaptive Computation and Machine Learning). MIT Press, Cambridge (2006)Google Scholar
  6. 6.
    Collobert, R., Weston, J.: A unified architecture for nlp: deep neural networks with multitask learning. In: ICML 2008, pp. 160–167 (2008)Google Scholar
  7. 7.
    Culotta, A., Sorensen, J.: Dependency tree kernels for relation extraction. In: ACL 2004, p. 423 (2004)Google Scholar
  8. 8.
    Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. JASIS 41(6), 391–407 (1990)CrossRefGoogle Scholar
  9. 9.
    Erkan, G., Ozgur, A., Radev, D.R.: Semi-supervised classification for extracting protein interaction sentences using dependency parsing. In: EMNLP-CoNLL 2007, pp. 228–237 (2007)Google Scholar
  10. 10.
    Gersho, A., Gray, R.M.: Vector quantization and signal compression, Norwell, MA, USA (1991)Google Scholar
  11. 11.
    Grandvalet, Y., Bengio, Y.: Semi-supervised learning by entropy minimization. In: NIPS 2005, pp. 529–536 (2005)Google Scholar
  12. 12.
    Koo, T., Collins, M.: Hidden-variable models for discriminative reranking. In: HLT 2005, pp. 507–514 (2005)Google Scholar
  13. 13.
    Krallinger, M., Morgan, A., Smith, L., Hirschman, L., Valencia, A., et al.: Evaluation of text-mining systems for biology: overview of the second biocreative community challenge. Genome Biol. 9(S2), S1 (2008)Google Scholar
  14. 14.
    Krallinger, M., Valencia, A., Hirschman, L.: Linking genes to literature: text mining, information extraction, and retrieval applications for biology. Genome Biol. 9(S2), S8 (2008)Google Scholar
  15. 15.
    Kuksa, P., Huang, P.H., Pavlovic, V.: Fast protein homology and fold detection with sparse spatial sample kernels. In: ICPR 2008 (2008)Google Scholar
  16. 16.
    Kuksa, P., Huang, P.H., Pavlovic, V.: Scalable algorithms for string kernels with inexact matching. In: NIPS, pp. 881–888 (2008)Google Scholar
  17. 17.
    Leslie, C., Kuang, R.: Fast string kernels using inexact matching for protein sequences. J. Mach. Learn. Res. 5, 1435–1455 (2004)MathSciNetGoogle Scholar
  18. 18.
    Leslie, C., Eskin, E., Noble, W.S.: The spectrum kernel: A string kernel for SVM protein classification. In: PSB, pp. 566–575 (2002)Google Scholar
  19. 19.
    Leslie, C., Eskin, E., Weston, J., Noble, W.S.: Mismatch string kernels for SVM protein classification. In: NIPS, pp. 1417–1424 (2002)Google Scholar
  20. 20.
    Miller, S., Guinness, J., Zamanian, A.: Name tagging with word clusters and discriminative training. In: HLT-NAACL 2004, pp. 337–342 (2004)Google Scholar
  21. 21.
    Miwa, M., Sætre, R., Miyao, Y., Tsujii, J.: A rich feature vector for protein-protein interaction extraction from multiple corpora. In: EMNLP, pp. 121–130 (2009)Google Scholar
  22. 22.
    Nigam, K., McCallum, A.K., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Mach. Learn. 39(2-3), 103–134 (2000)zbMATHCrossRefGoogle Scholar
  23. 23.
    Reichartz, F., Korte, H., Paass, G.: Dependency tree kernels for relation extraction from natural language text. In: ECML, pp. 270–285 (2009)Google Scholar
  24. 24.
    Rousu, J., Shawe-Taylor, J.: Efficient computation of gapped substring kernels on large alphabets. J. Mach. Learn. Res. 6, 1323–1344 (2005)Google Scholar
  25. 25.
    Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill Inc., New York (1986)Google Scholar
  26. 26.
    Segal, E., Koller, D., Ormoneit, D.: Probabilistic abstraction hierarchies. In: NIPS 2001 (2001)Google Scholar
  27. 27.
    Vishwanathan, S., Smola, A.: Fast kernels for string and tree matching, vol. 15, pp. 569–576. MIT Press, Cambridge (2002)Google Scholar
  28. 28.
    Weston, J., Leslie, C., Ie, E., Zhou, D., Elisseeff, A., Noble, W.S.: Semi-supervised protein classification using cluster kernels. Bioinformatics 21(15), 3241–3247 (2005)CrossRefGoogle Scholar
  29. 29.
    Zhou, D., He, Y.: Extracting interactions between proteins from the literature. J. Biomed. Inform. 41(2), 393–407 (2008)CrossRefGoogle Scholar
  30. 30.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML 2003, pp. 912–919 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Pavel Kuksa
    • 1
  • Yanjun Qi
    • 2
  • Bing Bai
    • 2
  • Ronan Collobert
    • 2
  • Jason Weston
    • 3
  • Vladimir Pavlovic
    • 1
  • Xia Ning
    • 4
  1. 1.Department of Computer ScienceRutgers UniversityUSA
  2. 2.NEC Labs AmericaPrincetonUSA
  3. 3.Google ResearchNew York CityUSA
  4. 4.Computer Science DepartmentUniversity of MinnesotaUSA

Personalised recommendations