Advertisement

An Experiment in Detection and Correction of Malapropisms Through the Web

  • Igor A. Bolshakov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3406)

Abstract

Malapropism is a type of semantic errors. It replaces one content word by another content word similar in sound but semantically incompatible with the context and thus destructing text cohesion. We propose to signal a malapropism when a pair of syntactically linked content words in a text exhibits the value of a specially defined Semantic Compatibility Index (SCI) lower than a predetermined threshold. SCI is computed through the web statistics of occurrences of words got together and apart. A malapropism detected, all possible candidates for correction of both words are taken from precompiled dictionaries of paronyms, i.e. words distant a letter or a few prefixes or suffixes from one another. Heuristic rules are proposed to retain only a few highly SCI-ranked candidates for the user’s decision. The experiment on mala-propism detection and correction is done for a hundred Russian text fragments—mainly from the web newswire—in both correct and falsified form, as well as for several hundreds of correction candidates. The raw statistics of occurrences is taken from the web searcher Yandex. Within certain limitations, the experiment gave very promising results.

Keywords

Word Form Content Word Editing Operation Semantic Error Orthographic Correction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bolshakov, I.A.: Getting one’s first million...Collocations. In: Gelbukh, A. (ed.) CICLing 2004. LNCS, vol. 2945, pp. 229–242. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Bolshakov, I.A., Gelbukh, A.: On Detection of Malapropisms by Multistage Collocation Testing. In: Düsterhöft, A., Talheim, B. (eds.) Proc. 8th Intern. Conference on Applications of Natural Language to Information Systems NLDB 2003, GI edn., Burg, Germany, Bonn, June 2003. LNI, vol. P-29, pp. 28–41 (2003)Google Scholar
  3. 3.
    Bolshakov, I.A., Gelbukh, A.: Paronyms for Accelerated Correction of Semantic Errors. International Journal on Information Theories & Applications 10, 198–204 (2003)Google Scholar
  4. 4.
    Gelbukh, A., Bolshakov, I.A.: On Correction of Semantic Errors in Natural Language Texts with a Dictionary of Literal Paronyms. In: Favela, J., Menasalvas, E., Chávez, E. (eds.) AWIC 2004. LNCS (LNAI), vol. 3034, pp. 105–114. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Gelbukh, A., Sidorov, G., Chanona-Hernández, L.: Compilation of a Spanish representative corpus. In: Gelbukh, A. (ed.) CICLing 2002. LNCS, vol. 2276, pp. 285–288. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Keller, F., Lapata, M.: Using the Web to Obtain Frequencies for Unseen Bigram. Computational linguistics 29(3), 459–484 (2003)CrossRefGoogle Scholar
  7. 7.
    Kilgarriff, A., Grefenstette, G.: Introduction to the Special Issue on the Web as Corpus. Computational linguistics 29(3), 333–347 (2003)CrossRefMathSciNetGoogle Scholar
  8. 8.
    The New Encyclopædia Britannica. Micropædia, vol. 7, Encyclopædia Britannica, Inc. (1998)Google Scholar
  9. 9.
    Hirst, G., St-Onge, D.: Lexical Chains as Representation of Context for Detection and Cor-rections of Malapropisms. In: Fellbaum, C. (ed.) WordNet: An Electronic Lexical Data-base, pp. 305–332. MIT Press, Cambridge (1998)Google Scholar
  10. 10.
    Hirst, G., Budanitsky, A.: Correcting Real-Word Spelling Errors by Restoring Lexical Co-hesion. In: Natural Language Engineering (2004) (to appear)Google Scholar
  11. 11.
    Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  12. 12.
    Mel’čuk, I.: Dependency Syntax: Theory and Practice. SUNY Press, NY (1988)Google Scholar
  13. 13.
    Pedersen, T.: A decision tree of bigrams is an accurate predictor of word senses. In: Proc. 2nd Annual Meeting of NAC ACL, Pittsburgh, PA, pp. 79–86 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Igor A. Bolshakov
    • 1
  1. 1.Center for Computing Research (CIC)National Polytechnic Institute (IPN)Mexico CityMexico

Personalised recommendations