Skip to main content
SpringerLink
Log in

Named Entity Recognition in Russian Using Multi-Task LSTM-CRF

  • Published:
Journal of Mathematical Sciences Aims and scope Submit manuscript

Named entity recognition (NER) is aimed at obtaining the important information from the unstructured data presented in the form of natural language texts. In this work, we investigate the efficiency of modern multi-task NER approaches on Russian language corpora by employing several different NER datasets and a dataset of part-of-speech (POS) tags. We apply a state-of-the-art neural architecture based on bidirectional LSTMs and conditional random fields. Convolutional neural networks were utilized to learn character-level features. We carry out extensive experimental evaluation over three standard datasets of news articles written in Russian. The proposed multi-task model achieve state-of-the-art results with an F1 score of 88.04% on Gareev’s dataset and an F1 score of 99.49% on Person-1000 dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al., “Tensorflow: a system for large-scale machine learning,” OSDI, 16, 265–283 (2016).

    Google Scholar 

  2. C. Adak, B. B. Chaudhuri, and M. Blumenstein, “Named entity recognition from unstructured handwritten document images,” in: 12th IAPR Workshop on Document Analysis Systems (DAS) (2016), pp. 375–380.

  3. L. T. Anh, M. Y. Arkhipov, and M. S. Burtsev, “Application of a hybrid bi-LSTM-CRF model to the task of Russian named entity recognition,” in: Communications in Computer and Information Science Book Series – CCIS, Vol. 789 (2017).

  4. L.T. Anh, M. Y. Arkhipov, and M. S. Burtsev, “Application of a hybrid bi-LSTM-CRF model to the task of Russian named entity recognition,” arXiv preprint arXiv:1709.09686 (2017).

  5. A. Y. Antonova and A. N. Soloviev, “Conditional random field models for the processing of Russian,” Communications of the ACM, 56, No. 6 (2013).

  6. M. Y. Arkhipov, M. S. Burtsev, and L. T. Anh, “Application of a hybrid bi-LSTM-CRF model to the task of Russian named entity recognition,” in: Conference on Artificial Intelligence and Natural Language, Springer, Cham (2017).

    Google Scholar 

  7. M. M. Brykina, A. V. Faynveyts, and S. Yu. Toldova, “Dictionary-based ambiguity resolution in Russian named entities recognition,” in: International Workshop on Computational Linguistics and its Applications (ed. A. Narin’yani), Vol. 1 (2013).

  8. R. Chalapathy, E. Z. Borzeshi, and M. Piccardi, “Bidirectional LSTM-CRF for clinical concept extraction,” arXiv preprint arXiv:1611.08373 (2016).

  9. J. P. C. Chiu and E. Nichols, “Named entity recognition with bidirectional LSTM-cnns,” Transactions of the Association for Computational Linguistics, 4, 357–370 (2016).

  10. L. G. Craidlin, “Program of allocation of Russian individualized nominal groups taglite,” Computational Linguistics and Intellectual Technologies Dialog (2005).

  11. D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in: 3rd International Conference for Learning Representations, San Diego (2014).

  12. C. Dong, J. Zhang, C. Zong, M. Hattori, and H. Di, “Character-based LSTM-CRF with radical-level features for chinese named entity recognition,” in: Natural Language Understanding and Intelligent Applications, Springer (2016), pp. 239–250.

    Chapter  Google Scholar 

  13. R. Gareev, M. Tkachenko, V. Solovyev, A. Simanovsky, and V. Ivanov, “Introducing baselines for Russian named entity recognition,” in: Computational Linguistics and Intelligent Text Processing (2013).

  14. A. Graves, S. Fernández, and J. Schmidhuber, “Bidirectional LSTM networks for improved phoneme classification and recognition,” in: Artificial Neural Networks: Formal Models and Their Applications – ICANN (2005).

  15. K. Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A search space odyssey,” IEEE Trans Neural Netw Learn Syst. doi: https://doi.org/10.1109/TNNLS.2016.2582924, 2016.

  16. Z. Huang, W. Xu, and K. Yu, “Bidirectional LSTM-CRF models for sequence tagging,” arXiv preprint arXiv:1508.01991 (2015).

  17. Kaggle, Predict Russian Universal Dependencies POS Tags (2017).

  18. G. Konoplich, E. Putin, A. Filchenkov, and R. Rybka, “Named entity recognition in Russian with word representation learned by a bidirectional language model,” AINL (2018).

  19. G. Konoplich, E. Putin, A. Filchenkov, and R. Rybka, “Named entity recognition in Russian with word representation learned by a bidirectional language model,” in: Conference on Artificial Intelligence and Natural Language, Springer (2018), pp. 48–58.

    Chapter  Google Scholar 

  20. J. Lafferty, A. McCallum, and F. Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data,” in: Proc. 18th International Conference on Machine Learning (2001).

  21. G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, “Neural architectures for named entity recognition,” in: Proc. 2016 NAACL (2016), pp. 260–270.

  22. X. Ma and E. Hovy, “End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF,” arXiv preprint arXiv:1603.01354 (2016).

  23. V. Malykh and A. Ozerin, “Reproducing Russian ner baseline quality without additional data,” CDUD@ CLA (2016), pp. 54–59.

  24. S. Misawa, M. Taniguchi, Y. Miura, and T. Ohkuma, “Character-based bidirectional LSTM-CRF with words and characters for japanese named entity recognition,” in Proc. 1st Workshop on Subword and Character Level Models in NLP (2017), pp. 97–102.

  25. V. Mozharova and N. Loukachevitch, “Two-stage approach in Russian named entity recognition,” in: Proc. 2016 International FRUCT Conference on Intelligence, Social Media and Web (ISMW FRUCT), IEEE (2016), pp. 1–6.

  26. A. V. Podobryaev, “Searching for person memories in news texts with the use of a model of conditional random fields,” RCDL (2013).

  27. B. Popov, A. Kiryakov, D. Ognyanoff, D. Manov, and A. Kirilov, “Kim – a semantic platform for information extraction and retrieval,” J. Natural Language Engineering, 10 (2004).

  28. R. M. Zavala, P. Martinez, and I. Segura-Bedmar, “A hybrid bi-LSTM-CRF model for knowledge recognition from ehealth documents,” TASS 2018: Workshop on Semantic Analysis at SEPLN (2018), pp. 65–70.

  29. R. Ivanitskiy, A. Shipilo, and L. Kovriguina, “Russian named entities recognition and classification using distributed word and phrase representations,” SIMBig (2016).

  30. A. V. Rubaylo and M. Y. Kosenko, “Software utilities for natural language information retrievial,” in: Almanac of Modern Science and Education, Vol. 12 (2016).

  31. E. Sheng, S. Miller, J.S. Ambite, and P. Natarajan, “A neural named entity recognition approach to biological entity identification,” in: Proc. BioCreative VI Workshop (2017), pp. 24–27.

  32. A. S. Starostin, V. V. Bocharov, S. V. Alexeeva, A. Bodrova, A. S. Chuchunkov, S. S. Dzhumaev, and M. A. Nikolaeva, “Evaluation of named entity recognition and fact extraction systems for Russian,” in: Annual International Conference Dialogue (2016).

  33. A. A. Sysoev and I. A. Andrianov, “Named entity recognition in Russian: the power of wiki-based approach,” in: Proc. International Conference Dialogue (2016), pp. 746–755.

  34. I. V. Trofimov,“Person name recognition in news articles based on the persons-1000/1111-f collections,” in: 16th All-Russian Scientific Conference Digital Libraries: Advanced Methods and Technologies, Digital Collections, RCDL (2014), pp. 217–221.

  35. E. Tutubalina and S. Nikolenko, “Combination of deep recurrent neural networks and conditional random fields for extracting adverse drug reactions from user reviews,” J. Healthcare Engineering, 2017, Article ID 9451342 (2017).

    Article  Google Scholar 

  36. N. A. Vlasova, E. A. Suleymanova, and I. V. Trofimov, “Report on Russian corpus for personal name retrieval,” in: Proceedings of Computational and Cognitive Linguistics TEL (2014).

  37. Q. Wei, T. Chen, R. Xu, Y. He, and L. Gui, “Disease named entity recognition by combining conditional random fields and bidirectional recurrent neural networks,” Database 2016 (2016).

Download references

Author information

Authors and Affiliations

  1. Kazan Federal University, Kazan, Russia

    D. Mazitov, I. Alimova & E. Tutubalina

Authors
  1. D. Mazitov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Alimova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. E. Tutubalina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. Mazitov.

Additional information

Published in Zapiski Nauchnykh Seminarov POMI, Vol. 499, 2021, pp. 222–235.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mazitov, D., Alimova, I. & Tutubalina, E. Named Entity Recognition in Russian Using Multi-Task LSTM-CRF. J Math Sci 273, 595–604 (2023). https://doi.org/10.1007/s10958-023-06521-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10958-023-06521-y

Search

Navigation