Abstract
The task of Chinese Spelling Check (CSC) is to detect and correct spelling errors in Chinese sentences. Since the scale of labeled CSC training set is quite small, we propose an unsupervised Chinese spelling correction framework based on detectors. Two kinds of detectors: Dec-Err and Dec-Eva, are proposed to leverage the contextual information to detect misspelled characters and evaluate the corrections respectively. Both detectors are fine-tuned with our proposed hybrid mask strategy. Dec-Eva is a transformer encoder based detector, of which we modify the attention connections to reuse the contextual information and parallel evaluate possible corrections. Compared with supervised and unsupervised state-of-the-art methods, experimental studies show that our method achieves competitive results. Further empirical studies reveal the efficiency and flexibility of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Pinyin is the romanization spelling system for the sounds of Chinese characters.
- 2.
- 3.
References
Chao, Y.C., Chang, C.H.: Automatic spelling correction for ASR corpus in traditional Chinese language using Seq2Seq models. In: 2020 International Computer Symposium (ICS), pp. 553–558. IEEE (2020)
Cheng, X., et al.: SpellGCN: incorporating phonological and visual similarities into language models for Chinese spelling check. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 871–881. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-main.81. https://aclanthology.org/2020.acl-main.81
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: ICLR (2020). https://openreview.net/pdf?id=r1xMH1BtvB
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/N19-1423. https://aclanthology.org/N19-1423
Hong, Y., Yu, X., He, N., Liu, N., Liu, J.: FASPell: a fast, adaptable, simple, powerful Chinese spell checker based on DAE-decoder paradigm. In: Proceedings of the 5th Workshop on Noisy User-Generated Text (W-NUT 2019), pp. 160–169 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980
Li, P.: uChecker: masked pretrained language models as unsupervised Chinese spelling checkers. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2812–2822. International Committee on Computational Linguistics, Gyeongju, Republic of Korea (2022). https://aclanthology.org/2022.coling-1.248
Liu, C.L., Lai, M.H., Chuang, Y.H., Lee, C.Y.: Visually and phonologically similar characters in incorrect simplified Chinese words. In: Coling 2010: Posters, pp. 739–747 (2010)
Liu, S., Yang, T., Yue, T., Zhang, F., Wang, D.: Plome: pre-training with misspelled knowledge for Chinese spelling correction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2991–3000 (2021)
Martins, B., Silva, M.J.: Spelling correction for search engine queries. In: Vicedo, J.L., Martínez-Barco, P., Muńoz, R., Saiz Noeda, M. (eds.) EsTAL 2004. LNCS (LNAI), vol. 3230, pp. 372–383. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30228-5_33
Nguyen, T.T.H., Jatowt, A., Coustaty, M., Doucet, A.: Survey of post-OCR processing approaches. ACM Comput. Surv. (CSUR) 54(6), 1–37 (2021)
Ramesh, D., Sanampudi, S.K.: An automated essay scoring systems: a systematic literature review. Artif. Intell. Rev. 1–33 (2021)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 6000–6010. Curran Associates Inc., Red Hook (2017)
Wang, D., Song, Y., Li, J., Han, J., Zhang, H.: A hybrid approach to automatic corpus generation for Chinese spelling check. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2517–2527 (2018)
Xie, W., et al.: Chinese spelling check system based on n-gram model. In: Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing, pp. 128–136 (2015)
Yeh, J.F., Li, S.F., Wu, M.R., Chen, W.Y., Su, M.C.: Chinese word spelling correction based on n-gram ranked inverted index list. In: Proceedings of the Seventh SIGHAN Workshop on Chinese Language Processing, pp. 43–48 (2013)
Yu, J., Li, Z.: Chinese spelling error detection and correction based on language model, pronunciation, and shape. In: Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pp. 220–223 (2014)
Zhang, R., et al.: Correcting Chinese spelling errors with phonetic pre-training. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2250–2261 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shao, F., Li, J. (2023). Dual-Detector: An Unsupervised Learning Framework for Chinese Spelling Check. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13938. Springer, Cham. https://doi.org/10.1007/978-3-031-33383-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-33383-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33382-8
Online ISBN: 978-3-031-33383-5
eBook Packages: Computer ScienceComputer Science (R0)