Real-Word Typo Detection
Context-sensitive spelling correction (CSSC) is a widely accepted and long studied formalization of the problem of finding and fixing contextually incorrect words. We argue that CSSC has its limitations as a model, and propose a weakened CSSC model (RWTD) to partially counter these limitations. We weaken the CSSC model by canceling its word-correction role. Thus, RWTD is focused solely on finding words that require correction. Once this is done, the actual correction process is performed by a human or a CSSC solution.
We propose a preliminary solution for RWTD model that differs from related CSSC work in several ways. The solution does not rely on a set of confusion lists and detects not only a limited set of confusion typos, but almost any class of typos. The solution offers a flexible trade-off between the time a human is willing to spend on the task and the quality of the proofreading. It does not require POS tagging and may be applied seamlessly to different languages. Experiment running times prove to be acceptable for real-world applications.
We report Brown corpus real-word typos that were exposed by implementing our solution. We also discuss experiments in applying the solution to other real-world test texts and demonstrate improved false positive and hit rates.
KeywordsWord Pair News Article Optical Character Recognition Training Corpus Input Text
Unable to display preview. Download preview PDF.
- 1.Golding, A.R., Roth, D.: A winnow-based approach to context-sensitive spelling correction. Machine Learning 34(1) (February 1999)Google Scholar
- 4.Reynaert, M.: All, and only, the errors: more complete and consistent spelling and ocr-error correction evaluation. In: Proceedings of the Sixth International Language Resources and Evaluation (LREC 2008), Marrakech, Morocco (2008)Google Scholar
- 5.Bolshakov, I.A., Bolshakova, E.I., Kotlyarov, A.P., Gelbukh, A.F.: Various criteria of collocation cohesion in internet: Comparison of resolving power. In: Computational Linguistics and Intelligent Text Processing, Haifa, Israel (2008)Google Scholar
- 6.Asonov, D.: Real-word typo detection: Supplementary material (2009), http://www.fastpl.com/pubs/nldb09supm.pdf
- 7.Hirst, G.: An evaluation of the contextual spelling checker of microsoft office word 2007 (2008)Google Scholar
- 8.Mitton, R.: Spellchecking by computer. Journal of the Simplified Spelling Society 20(1) (1996)Google Scholar
- 10.Morris, R., Cherry, L.L.: Computer detection of typographical errors. IEEE Transactions on Professional Communication 18(1) (1975)Google Scholar
- 11.Bolshakova, E., Bolshakov, I., Kotlyarov, A.: Experiments in detection and correction of russian malapropisms by means of the web. International Journal Information Theories and Applications 12 (2006)Google Scholar
- 12.Fossati, D., Eugenio, B.D.: I saw tree trees in the park: How to correct real-word spelling mistakes. In: Proceedings of the Sixth International Language Resources and Evaluation (LREC 2008), Marrakech, Morocco (2008)Google Scholar