Automatic Evaluation of Quality of an Explanatory Dictionary by Comparison of Word Senses

  • Alexander Gelbukh
  • Grigori Sidorov
  • SangYong Han
  • Liliana Chanona-Hernandez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2890)


Words in the explanatory dictionary have different meanings (senses) described using natural language definitions. If the definitions of two senses of the same word are too similar, it is difficult to grasp the difference and thus it is difficult to judge which of the two senses is intended in a particular contexts, especially when such a decision is to be made automatically as in the task of automatic word sense disambiguation. We suggest a method of formal evaluation of this aspect of quality of an explanatory dictionary by calculating the similarity of different senses of the same word. We calculate the similarity between two given senses as the relative number of equal or synonymous words in their definitions. In addition to the general assessment of the dictionary, the individual suspicious definitions are reported for possible improvement. In our experiments we used the Anaya explanatory dictionary of Spanish. Our experiments show that there are about 10% of substantially similar definitions in this dictionary, which indicates rather low quality.


Word Sense Word Sense Disambiguation Automatic Evaluation Branch Bank Sense Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: Assessing the Effectiveness of the DAML Ontologies for the Semantic Web. In: Proc. NLDB-2003, Applications of Natural Language to Information Systems, Berlin. Lecture Notes in Informatics (2003)Google Scholar
  2. 2.
    Gelbukh, A., Sidorov, G.: Morphological Analysis of Inflective Languages Through Generation. J. Procesamiento de Lenguaje Natural (29), 105–112 (2002)Google Scholar
  3. 3.
    Jiang, J.J., Conrad, D.W.: From object comparison to semantic similarity. In: Proc. of Pacling 1999 (Pacific association for computational linguistics), Waterloo, Canada, August 1999, pp. 256–263 (1999)Google Scholar
  4. 4.
    Lesk, M.: Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from a ice cream cone. In: Proceedings of SIGDOC 1986 (1986)Google Scholar
  5. 5.
    Magnini, B., Strapparava, C.: Experiments in Word Domain Disambiguation for Parallel Texts. In: Proceedings of the ACL Workshop on Word Senses and Multilinguality, Hong Kong, China (2000)Google Scholar
  6. 6.
    Manning, C.D., Shutze, H.: Foundations of statistical natural language processing, p. 680. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  7. 7.
    Suárez, A., Palomar, M.: Word Sense vs. Word Domain Disambiguation: a Maximum Entropy approach. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2002. LNCS (LNAI), vol. 2448, p. 131. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)Google Scholar
  9. 9.
    Rasmussen, E.: Clustering algorithms. In: Frakes, W.B., Baeza-Yates, R. (eds.) Information Retrieval: Data Structures and Algorithms, pp. 419–442. Prentice Hall, Upper Saddle River (1992)Google Scholar
  10. 10.
    Sidorov, G., Gelbukh, A.: Word sense disambiguation in a Spanish explanatory dictionary. In: Proc. of TALN 2001, Traitement Automatique du Langage Naturel, Tours, France, July 2-5, pp. 398–402 (2001)Google Scholar
  11. 11.
    Fellbaum, C. (ed.): WordNet: an electronic lexical database, p. 423. MIT, Cambridge (1998)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Alexander Gelbukh
    • 1
    • 2
  • Grigori Sidorov
    • 1
  • SangYong Han
    • 2
  • Liliana Chanona-Hernandez
    • 3
  1. 1.National Polytechnic Institute (IPN)Center for Computing Research (CIC)Mexico CityMexico
  2. 2.Department of Computer Science and EngineeringChung-Ang UniversitySeoulKorea
  3. 3.Department of Computer ScienceLas Americas UniversityMexico CityMexico

Personalised recommendations