Thesaurus Extension Using Web Search Engines

  • Robert Meusel
  • Mathias Niepert
  • Kai Eckert
  • Heiner Stuckenschmidt
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6102)


Maintaining and extending large thesauri is an important challenge facing digital libraries and IT businesses alike. In this paper we describe a method building on and extending existing methods from the areas of thesaurus maintenance, natural language processing, and machine learning to (a) extract a set of novel candidate concepts from text corpora and (b) to generate a small ranked list of suggestions for the position of these concept in an existing thesaurus. Based on a modification of the standard tf-idf term weighting we extract relevant concept candidates from a document corpus. We then apply a pattern-based machine learning approach on content extracted from web search engine snippets to determine the type of relation between the candidate terms and existing thesaurus concepts. The approach is evaluated with a large-scale experiment using the MeSH and WordNet thesauri as testbed.


Support Vector Machine Text Corpus Decision Tree Algorithm Linear Support Vector Machine Document Corpus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bollegala, D., Matsuo, Y., Ishizuka, M.: An integrated approach to measuring semantic similarity between words using information available on the web. In: Proceedings of HLT-NAACL, pp. 340–347 (2007)Google Scholar
  2. 2.
    Bollegala, D., Matsuo, Y., Ishizuka, M.: Measuring semantic similarity between words using web search engines. In: Proceedings of WWW, pp. 757–766 (2007)Google Scholar
  3. 3.
    Curran, J.R.: Ensemble methods for automatic thesaurus extraction. In: Proceedings of ACL, pp. 222–229 (2002)Google Scholar
  4. 4.
    Eckert, K., Niepert, M., Niemann, C., Buckner, C., Allen, C., Stuckenschmidt, H.: Crowdsourcing the assembly of concept hierarchies. In: Proceedings of JCDL (2010)Google Scholar
  5. 5.
    Gillam, L., Tariq, M., Ahmad, K.: Terminology and the construction of ontology. Terminology 11, 55–81 (2005)CrossRefGoogle Scholar
  6. 6.
    Grefenstette, G.: Explorations in Automatic Thesaurus Discovery. Springer, Heidelberg (1994)zbMATHGoogle Scholar
  7. 7.
    Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: Proceedings of the Fourteenth International Conference on Computational Linguistics, Nantes, France (1992)Google Scholar
  8. 8.
    Kageura, K., Tsuji, K., Aizawa, A.N.: Automatic thesaurus generation through multiple filtering. In: Proceedings COLING, pp. 397–403 (2000)Google Scholar
  9. 9.
    Kaji, N., Kitsuregawa, M.: Using hidden markov random fields to combine distributional and pattern-based word clustering. In: Proceedings of COLING, pp. 401–408 (2008)Google Scholar
  10. 10.
    Kermanidis, M.M.K.L., Thanopoulos, A., Fakotakis, N.: Eksairesis: A domain-adaptable system for ontology building from unstructured text. In: Proceedings of LREC (2008)Google Scholar
  11. 11.
    Lin, D.: An information-theoretic definition of similarity. In: Proceedings of ICML, pp. 296–304 (1998)Google Scholar
  12. 12.
    Matsuo, Y., Sakaki, T., Uchiyama, K., Ishizuka, M.: Graph-based word clustering using a web search engine. In: Proceedings of EMNLP, pp. 542–550 (2006)Google Scholar
  13. 13.
    Nguyen, D.P.T., Matsuo, Y., Ishizuka, M.: Exploiting syntactic and semantic information for relation extraction from wikipedia. In: Proceedings of IJCAI (2007)Google Scholar
  14. 14.
    Niepert, M., Buckner, C., Allen, C.: A dynamic ontology for a dynamic reference work. In: Proceedings of JCDL, pp. 288–297 (2007)Google Scholar
  15. 15.
    Turney, P.D.: Mining the web for synonyms: Pmi-ir versus lsa on toefl. In: Proceedings of EMCL, pp. 491–502 (2001)Google Scholar
  16. 16.
    van der Plas, L., Tiedemann, J.: Finding synonyms using automatic word alignment and measures of distributional similarity. In: Proceedings of COLING, pp. 866–873 (2006)Google Scholar
  17. 17.
    Witschel, H.F.: Using decision trees and text mining techniques for extending taxonomies. In: Proceedings of the Workshop on Learning and Extending Lexical Ontologies by Using Machine Learning Methods (2005)Google Scholar
  18. 18.
    Wu, H., Zhou, M.: Optimizing synonym extraction using monolingual and bilingual resources. In: Proceedings of the second international workshop on Paraphrasing, pp. 72–79 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Robert Meusel
    • 1
  • Mathias Niepert
    • 1
  • Kai Eckert
    • 1
  • Heiner Stuckenschmidt
    • 1
  1. 1.KR & KM Research GroupUniversity of MannheimGermany

Personalised recommendations