Advertisement

Boosting Applied to Word Sense Disambiguation

  • Gerard Escudero
  • Lluís Màrquez
  • German Rigau
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1810)

Abstract

In this paper Schapire and Singer’s AdaBoost.MH boosting algorithm is applied to the Word Sense Disambiguation (WSD) problem. Initial experiments on a set of 15 selected polysemous words show that the boosting approach surpasses Naive Bayes and Exemplar-based approaches, which represent state-of-the-art accuracy on supervised WSD. In order to make boosting practical for a real learning domain of thousands of words, several ways of accelerating the algorithm by reducing the feature space are studied. The best variant, which we call LazyBoosting, is tested on the largest sense-tagged corpus available containing 192,800 examples of the 191 most frequent and ambiguous English words. Again, boosting compares favourably to the other benchmark algorithms.

Keywords

Natural Language Processing Weak Learner Weak Hypothesis Annotate Corpus Polysemous Word 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Abney, S., Schapire, R.E. and Singer, Y.: Boosting Applied to Tagging and PP-attachment. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, 1999. 130, 132, 134, 139Google Scholar
  2. 2.
    Bauer, E. and Kohavi, R.: An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting and Variants. Machine Learning Journal. Special issue on IMLM for Improving and Scaling Machine Learning Algorithms, 1999. 130Google Scholar
  3. 3.
    Breiman, L.: Arcing Classifiers. The Annals of Statistics, 26(3), 1998.Google Scholar
  4. 4.
    Duda, R. O. and Hart, P. E.: Pattern Classification and Scene Analysis. Wiley, New York, 1973. 134zbMATHGoogle Scholar
  5. 5.
    Dietterich, T.G.: An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization. Machine Learning (to appear). 130Google Scholar
  6. 6.
    Engelson, S.P. and Dagan, I.: Minimizing Manual Annotation Cost in Supervised Training from Corpora. In S. Wermter, E. Riloff and G. Scheler, editors, Connectionist, Statistical an Symbolic Approaches to Learning for Natural Language Processing, LNAI, 1040. Springer, 1996. 130Google Scholar
  7. 7.
    Escudero, G., Màrquez, L. and Rigau, G. Boosting Applied to Word Sense Disambiguation. Technical Report LSI-00-3-R, LSI Department, UPC, 2000. 134, 139Google Scholar
  8. 8.
    Freund, Y. and Schapire, R.E.: Experiments with a New Boosting Algorithm. In Procs. of the 13th International Conference on Machine Learning, ICML, 1996. 130Google Scholar
  9. 9.
    Freund, Y. and Schapire, R.E.: A Decision-theoretic Generalization of On-line Learning and an Application to Boosting. Journal of Computer and System Sciences, 55(1), 1997. 130Google Scholar
  10. 10.
    Fujii, A., Inui, K., Tokunaga, T. and Tanaka, H.: Selective Sampling for Example-based Word Sense Disambiguation. Computational Linguistics, 24(4), ACL, 1998. 130Google Scholar
  11. 11.
    Ide, N. and Véronis, J.: Introduction to the Special Issue on Word Sense Disambiguation: The State of the Art. Computational Linguistics, 24(1), ACL, 1998. 129Google Scholar
  12. 12.
    Leacock, C., Chodorow, M. and Miller, G.A.: Using Corpus Statistics and Word-Net Relations for Sense Identification. Computational Linguistics, 24(1), ACL, 1998. 130Google Scholar
  13. 13.
    López de Mántaras, R.: A Distance-based Attribute Selection Measure for Decision Tree Induction. Machine Learning, 6(1), 1991. 136Google Scholar
  14. 14.
    Màrquez, L.: Part-of-speech Tagging: A Machine Learning Approach based on Decision Trees. Phd. Thesis, LSI Department, UPC, 1999. 137Google Scholar
  15. 15.
    Mihalcea, R. and Moldovan, I.: An Automatic Method for Generating Sense Tagged Corpora. In Proceedings of the 16th National Conference on Artificial Intelligence, AAAI, 1999. 130Google Scholar
  16. 16.
    Miller, G.A., Beckwith, R., Fellbaum, C., Gross, D. and Miller, K.: Five Papers on WordNet. Special Issue of International Journal of Lexicography, 3(4), 1990. 130Google Scholar
  17. 17.
    Mooney, R.J.: Comparative Experiments on Disambiguating Word Senses: An Illustration of the Role of Bias in Machine Learning. In Proceedings of the 1st Conference on Empirical Methods in Natural Language Processing, EMNLP, 1996. 130Google Scholar
  18. 18.
    Ng, H.T. and Lee, H.B.: Integrating Multiple Knowledge Sources to Disambiguate Word Senses: An Exemplar-based Approach. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics, ACL, 1996. 133Google Scholar
  19. 19.
    Ng, H.T.: Exemplar-based Word Sense Disambiguation: Some Recent Improvements. In Proceedings of the 2nd Conference on Empirical Methods in Natural Language Processing, EMNLP, 1997. 130, 134, 135Google Scholar
  20. 20.
    Ng, H.T.: Getting Serious about Word Sense Disambiguation. In Proceedings of the SIGLEX Workshop “Tagging Text with Lexical Semantics: Why, What and How”, 1997. 130Google Scholar
  21. 21.
    Ng, H.T., Chung, Y. L. and Shou, K. F.: A Case Study on Inter-Annotation Agreement for WSD. In Proceedings of the SIGLEX Workshop “Standardizing Lexical Resources”, Maryland, USA, 1999. 133Google Scholar
  22. 22.
    Pedersen, T. and Bruce, R.: Knowledge Lean Word-Sense Disambiguation. In Proceedings of the 15th National Conference on Artificial Intelligence, 1998. 130Google Scholar
  23. 23.
    Quinlan, J.R.: Bagging, Boosting and C4.5. In Proceedings of the 13th National Conference on Artificial Intelligence, AAAI, 1996. 130Google Scholar
  24. 24.
    Samuel, K.: Lazy Transformation-Based Learning. In Proceedings of the 11th International Florida AI Research Symposium Conference, 1998. 137Google Scholar
  25. 25.
    Schapire, R.E. and Singer, Y.: Improved Boosting Algorithms Using Confidence-rated Predictions. Machine Learning (to appear). 130, 131, 132, 133Google Scholar
  26. 26.
    Schapire, R.E. and Singer, Y.: BoosTexter: A Boosting-based System for Text Categorization. Machine Learning (to appear). 130, 131, 133, 139Google Scholar
  27. 27.
    Towell, G. and Voorhees, E.M.: Disambiguating Highly Ambiguous Words. Computational Linguistics, 24(1), ACL, 1998. 130Google Scholar
  28. 28.
    Yarowsky, D.: Decision Lists for Lexical Ambiguity Resolution: Application to Accent Restoration in Spanish and French. In Proceedings of the 32nd annual Meeting of the Association for Computational Linguistics, ACL, 1994. 130Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Gerard Escudero
    • 1
  • Lluís Màrquez
    • 1
  • German Rigau
    • 1
  1. 1.TALP Research Center, LSI DepartmentUniversitat Politècnica de Catalunya (UPC)Barcelona. Catalonia

Personalised recommendations