Skip to main content
SpringerLink
Log in
Book cover

International Conference on Text, Speech, and Dialogue

TSD 2017: Text, Speech, and Dialogue pp 473–481Cite as

Ship-LemmaTagger: Building an NLP Toolkit for a Peruvian Native Language

  • Conference paper
  • First Online:

  • 1537 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10415))

Abstract

Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Available in: chana.inf.pucp.edu.pe/chanot.

  2. 2.

    Available in: chana.inf.pucp.edu.pe/resources.

References

  1. Acosta, S., Natalia, K., Huamancayo Curi, E., Mori Clement, M., Carbajal Solis, V.: Documento nacional de lenguas originarias del Perú (2013)

    Google Scholar 

  2. Daelemans, W., Groenewald, H.J., Van Huyssteen, G.B.: Prototype-Based Active Learning for Lemmatization (2009)

    Google Scholar 

  3. Ekbal, A., Bandyopadhyay, S.: Part of speech tagging in Bengali using support vector machine. In: International Conference on Information Technology 2008, ICIT 2008, pp. 106–111. IEEE (2008)

    Google Scholar 

  4. Gupta, V., Joshi, N., Mathur, I.: Design and development of a rule-based Urdu lemmatizer. In: Proceedings of International Conference on ICT for Sustainable Development, pp. 161–169. Springer (2016)

    Google Scholar 

  5. Jurafsky, D., Martin, J.H.: Speech and Language Processing, vol. 3. Pearson, London (2014)

    Google Scholar 

  6. Juršic, M., Mozetic, I., Erjavec, T., Lavrac, N.: Lemmagen: multilingual lemmatisation with induced ripple-down rules. J. Univers. Comput. Sci. 16(9), 1190–1214 (2010)

    Google Scholar 

  7. Khaltar, B.O., Fujii, A.: A lemmatization method for mongolian and its application to indexing for information retrieval. Inf. Process. Manag. 45(4), 438–451 (2009)

    Article  Google Scholar 

  8. Ministerio de Educación del Perú: Minedu oficializa alfabetos de 24 lenguas originarias a ser utilizados por todas las entidades públicas. http://www.minedu.gob.pe/n/noticia.php?id=33082. Accessed 31 Mar 2016

  9. Nivre, J., de Marneffe, M.C., Ginter, F., Goldberg, Y., Hajic, J., Manning, C.D., McDonald, R., Petrov, S., Pyysalo, S., Silveira, N., et al.: Universal dependencies v1: A multilingual treebank collection. In: Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pp. 1659–1666 (2016)

    Google Scholar 

  10. Paul, A., Purkayastha, B.S., Sarkar, S.: Hidden Markov model based part of speech tagging for Nepali language. In: 2015 International Symposium on Advanced Computing and Communication (ISACC), pp. 149–156. IEEE (2015)

    Google Scholar 

  11. Singh, S., Jha, G.N.: Statistical tagger for Bhojpuri (employing support vector machine). In: 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1524–1529. IEEE (2015)

    Google Scholar 

  12. Valenzuela, P.: Transitivity in Shipibo-Konibo grammar. Ph.D. thesis, University of Oregon (2003)

    Google Scholar 

Download references

Acknowledgments

For this study, the authors appreciate the linguistic team effort that made possible the corpus annotation, and also acknowledge the support of the “Consejo Nacional de Ciencia, Tecnología e Innovación Tecnológica” (CONCYTEC Perú) under the contract 225-2015-FONDECYT.

Author information

Authors and Affiliations

  1. Facultad de Ciencias e Ingeniería, Pontificia Universidad Católica del Perú, Lima, Peru

    José Pereira-Noriega

  2. Research Group on Pattern Recognition and Applied Artificial Intelligence, Departamento de Ingeniería, Pontificia Universidad Católica del Perú, Lima, Peru

    Rodolfo Mercado-Gonzales, Andrés Melgar, Marco Sobrevilla-Cabezudo & Arturo Oncevay-Marcos

Authors
  1. José Pereira-Noriega
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodolfo Mercado-Gonzales
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrés Melgar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Sobrevilla-Cabezudo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arturo Oncevay-Marcos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arturo Oncevay-Marcos .

Editor information

Editors and Affiliations

  1. University of West Bohemia, Pilsen, Czech Republic

    Kamil Ekštein

  2. University of West Bohemia, Pilsen, Czech Republic

    Václav Matoušek

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Pereira-Noriega, J., Mercado-Gonzales, R., Melgar, A., Sobrevilla-Cabezudo, M., Oncevay-Marcos, A. (2017). Ship-LemmaTagger: Building an NLP Toolkit for a Peruvian Native Language. In: Ekštein, K., Matoušek, V. (eds) Text, Speech, and Dialogue. TSD 2017. Lecture Notes in Computer Science(), vol 10415. Springer, Cham. https://doi.org/10.1007/978-3-319-64206-2_53

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64206-2_53

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64205-5

  • Online ISBN: 978-3-319-64206-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation