LDA in Character-LSTM-CRF Named Entity Recognition

  • Miloslav KonopíkEmail author
  • Ondřej Pražák
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11107)


In this paper, we present a NER system based upon deep learning models with character sequence encoding and word sequence encoding in LSTM layers. The results are boosted with LDA topic models and linear-chain CRF sequence tagging. We reach the new state-of-the-art performance in NER of 81.77 F-measure for Czech and 85.91 F-measure Spanish.


Named entity recognition LSTM LDA Tensorflow 



This work was supported by Ministry of Education, Youth and Sports of the Czech Republic, institutional research support (1311) and by the UWB grant no. SGS-2013-029 Advanced computing and information systems. Access to the MetaCentrum computing facilities provided under the program “Projects of Large Infrastructure for Research, Development, and Innovations” LM2010005, funded by the Ministry of Education, Youth, and Sports of the Czech Republic, is highly appreciated.


  1. 1.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015)., Software available from
  2. 2.
    Agerri, R., Rigau, G.: Robust multilingual named entity recognition with shallow semi-supervised features. Artif. Intell. 238, 63–82 (2016)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Blei, D.M., Ng, A.Y., Jordan, M.I., Lafferty, J.: Latent dirichlet allocation. J. Mach. Learn. Res. 3 (2003)Google Scholar
  4. 4.
    Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016). Scholar
  5. 5.
    Konkol, M., Brychcn, T., Konopk, M.: Latent semantics in named entity recognition. Expert Syst. Appl. 42(7), 3470–3479 (2015)., Scholar
  6. 6.
    Konkol, M., Konopík, M.: CRF-based Czech named entity recognizer and consolidation of Czech NER research. In: Habernal, I., Matoušek, V. (eds.) TSD 2013. LNCS (LNAI), vol. 8082, pp. 153–160. Springer, Heidelberg (2013). Scholar
  7. 7.
    Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics (2016).,
  8. 8.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 7–12 August 2016, vol. 1: Long Papers. The Association for Computer Linguistics (2016).
  9. 9.
    McCallum, A.K.: Mallet: a machine learning for language toolkit (2002).
  10. 10.
    Nallapati, R., Surdeanu, M., Manning, C.: Blind domain transfer for named entity recognition using generative latent topic models. In: Proceedings of the NIPS 2010 Workshop on Transfer Learning Via Rich Generative Models, pp. 281–289 (2010)Google Scholar
  11. 11.
    dos Santos, C.N., Guimarães, V.: Boosting named entity recognition with neural character embeddings. In: Duan, X., Banchs, R.E., Zhang, M., Li, H., Kumaran, A. (eds.) Proceedings of the Fifth Named Entity Workshop, NEWS@ACL 2015, Beijing, China, 31 July 2015, pp. 25–33. Association for Computational Linguistics (2015).
  12. 12.
    Straková, J., Straka, M., Hajič, J.: Neural networks for featureless named entity recognition in Czech. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2016. LNCS (LNAI), vol. 9924, pp. 173–181. Springer, Cham (2016). Scholar
  13. 13.
    Tjong Kim Sang, E.F.: Introduction to the CoNLL-2002 shared task: language-independent named entity recognition. In: Proceedings of CoNLL 2002, Taipei, Taiwan, pp. 155–158 (2002)Google Scholar
  14. 14.
    Ševčíková, M., Žabokrtský, Z., Krůza, O.: Named entities in Czech: annotating data and developing NE tagger. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 188–195. Springer, Heidelberg (2007). Scholar
  15. 15.
    Yang, Z., Salakhutdinov, R., Cohen, W.W.: Multi-task cross-lingual sequence tagging from scratch. CoRR abs/1603.06270 (2016).

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer Science and Engineering, Faculty of Applied SciencesUniversity of West BohemiaPlzeňCzech Republic

Personalised recommendations