Automatic Evaluation of Quality of an Explanatory Dictionary by Comparison of Word Senses
Words in the explanatory dictionary have different meanings (senses) described using natural language definitions. If the definitions of two senses of the same word are too similar, it is difficult to grasp the difference and thus it is difficult to judge which of the two senses is intended in a particular contexts, especially when such a decision is to be made automatically as in the task of automatic word sense disambiguation. We suggest a method of formal evaluation of this aspect of quality of an explanatory dictionary by calculating the similarity of different senses of the same word. We calculate the similarity between two given senses as the relative number of equal or synonymous words in their definitions. In addition to the general assessment of the dictionary, the individual suspicious definitions are reported for possible improvement. In our experiments we used the Anaya explanatory dictionary of Spanish. Our experiments show that there are about 10% of substantially similar definitions in this dictionary, which indicates rather low quality.
KeywordsWord Sense Word Sense Disambiguation Automatic Evaluation Branch Bank Sense Pair
Unable to display preview. Download preview PDF.
- 1.Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: Assessing the Effectiveness of the DAML Ontologies for the Semantic Web. In: Proc. NLDB-2003, Applications of Natural Language to Information Systems, Berlin. Lecture Notes in Informatics (2003)Google Scholar
- 2.Gelbukh, A., Sidorov, G.: Morphological Analysis of Inflective Languages Through Generation. J. Procesamiento de Lenguaje Natural (29), 105–112 (2002)Google Scholar
- 3.Jiang, J.J., Conrad, D.W.: From object comparison to semantic similarity. In: Proc. of Pacling 1999 (Pacific association for computational linguistics), Waterloo, Canada, August 1999, pp. 256–263 (1999)Google Scholar
- 4.Lesk, M.: Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from a ice cream cone. In: Proceedings of SIGDOC 1986 (1986)Google Scholar
- 5.Magnini, B., Strapparava, C.: Experiments in Word Domain Disambiguation for Parallel Texts. In: Proceedings of the ACL Workshop on Word Senses and Multilinguality, Hong Kong, China (2000)Google Scholar
- 8.Porter, M.F.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)Google Scholar
- 9.Rasmussen, E.: Clustering algorithms. In: Frakes, W.B., Baeza-Yates, R. (eds.) Information Retrieval: Data Structures and Algorithms, pp. 419–442. Prentice Hall, Upper Saddle River (1992)Google Scholar
- 10.Sidorov, G., Gelbukh, A.: Word sense disambiguation in a Spanish explanatory dictionary. In: Proc. of TALN 2001, Traitement Automatique du Langage Naturel, Tours, France, July 2-5, pp. 398–402 (2001)Google Scholar