Abstract
The process of assigning words in a corpus to the corresponding specified tags based on the context and its definition is part of speech (POS) tagging. It is always been a big challenge yet a very important task in language processing. The task of a part of speech tagging is more challenging in low resource languages, for example, Mizo language. This paper presents the development of a data-driven part of speech tagging system for the Mizo language. This research work includes creating of tagset and annotated Mizo corpus, development of stochastic based taggers such as unigram and bigram Hidden Markov model. The highest accuracy obtained using the proposed unigram and bigram Hidden Markov Model (HMM) based part of speech (POS) taggers are 70.61% and 75.19% respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ethnologue, languages of the world. https://www.ethnologue.com/ethnoblog/gary-simons/welcome-23rd-edition. Last accessed 2020/11/01
Wikipedia. The free encyclopedia. https://en.wikipedia.org/wiki/List_of_endangered_languages_in_India. Last accessed 2020/11/01
Jurafsky D (2000) Speech & language processing, 2nd edn. Pearson Education, India
Kupiec J (1992) Robust part-of-speech tagging using a hidden Markov model. Comput Speech Lang 6(3):225–242
Al Shamsi F, Guessoum A (2006) A hidden Markov model-based POS tagger for Arabic. In: Proceeding of the 8th international conference on the statistical analysis of textual data, France, pp 31–42
Tasharofi S, Raja F, Oroumchian F, Rahgozar M (2007) Evaluation of statistical part of speech tagging of Persian text. In: 2007 9th international symposium on signal processing and its applications. IEEE, pp 1–4
Anwar W, Wang X, Li L, Wang XL (2007) A statistical based part of speech tagger for Urdu language. In: 2007 international conference on machine learning and cybernetics, vol 6. IEEE, pp 3418–3424
Kadim A, Lazrek A (2018) Parallel HMM-based approach for Arabic part of speech tagging. Int Arab J Inf Technol 15(2):341–351
Cahyani DE, Vindiyanto MJ (2019) Indonesian part of speech tagging using hidden Markov model–Ngram & Viterbi. In: 2019 4th international conference on information technology, information systems and electrical engineering (ICITISEE). IEEE, pp 353–358
Joshi N, Darbari H, Mathur I (2013) HMM based POS tagger for Hindi. In: Proceeding of 2013 international conference on artificial intelligence, soft computing (AISC-2013), pp 341–349
Linn ZZ, Patil PB (2019) Part of speech tagging for Kayah language using hidden Markov model. In: 2019 4th international conference on electrical, electronics, communication, computer technologies and optimization techniques (ICEECCOT). IEEE, pp 228–233
Singh J, Joshi N, Mathur I (2013) Development of Marathi part of speech tagger using statistical approach. In: Advances in computing, communications and informatics (ICACCI)
Mohammed S (2020) Using machine learning to build POS tagger for under-resourced language: the case of Somali. Int J Inf Technol 12:717–729
Wikipedia. The free encyclopedia. https://en.wikipedia.org/wiki/Part-of-speech_tagging. Last accessed 2020/11/01
Yimin L, Degen H (2005) Chinese part-of-speech tagging based on full second-order Hidden Markov model. Comput Eng 10
Daimary SK, Goyal V, Barbora M, Singh U (2018) Development of part of speech tagger for Assamese using HMM. Int J Synth Emotions (IJSE) 9(1):23–32
Bandyopadhyay S, Ekbal A (2007) HMM based POS tagger and rule-based Chunker for Bengali. In: Advances in pattern recognition, pp 384–390
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Nunsanga, M.V.L., Pakray, P., Lalngaihtuaha, M., Lolit Kumar Singh, L. (2022). Stochastic Based Part of Speech Tagging in Mizo Language: Unigram and Bigram Hidden Markov Model. In: Patgiri, R., Bandyopadhyay, S., Borah, M.D., Emilia Balas, V. (eds) Edge Analytics. Lecture Notes in Electrical Engineering, vol 869. Springer, Singapore. https://doi.org/10.1007/978-981-19-0019-8_53
Download citation
DOI: https://doi.org/10.1007/978-981-19-0019-8_53
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-0018-1
Online ISBN: 978-981-19-0019-8
eBook Packages: EngineeringEngineering (R0)