Advertisement

LDA in Character-LSTM-CRF Named Entity Recognition

  • Miloslav Konopík
  • Ondřej Pražák
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11107)

Abstract

In this paper, we present a NER system based upon deep learning models with character sequence encoding and word sequence encoding in LSTM layers. The results are boosted with LDA topic models and linear-chain CRF sequence tagging. We reach the new state-of-the-art performance in NER of 81.77 F-measure for Czech and 85.91 F-measure Spanish.

Keywords

Named entity recognition LSTM LDA Tensorflow 

Notes

Acknowledgements

This work was supported by Ministry of Education, Youth and Sports of the Czech Republic, institutional research support (1311) and by the UWB grant no. SGS-2013-029 Advanced computing and information systems. Access to the MetaCentrum computing facilities provided under the program “Projects of Large Infrastructure for Research, Development, and Innovations” LM2010005, funded by the Ministry of Education, Youth, and Sports of the Czech Republic, is highly appreciated.

References

  1. 1.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, Software available from tensorflow.org
  2. 2.
    Agerri, R., Rigau, G.: Robust multilingual named entity recognition with shallow semi-supervised features. Artif. Intell. 238, 63–82 (2016)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Blei, D.M., Ng, A.Y., Jordan, M.I., Lafferty, J.: Latent dirichlet allocation. J. Mach. Learn. Res. 3 (2003)Google Scholar
  4. 4.
    Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. Linguist. 4, 357–370 (2016). http://aclweb.org/anthology/Q16-1026
  5. 5.
    Konkol, M., Brychcn, T., Konopk, M.: Latent semantics in named entity recognition. Expert Syst. Appl. 42(7), 3470–3479 (2015).  https://doi.org/10.1016/j.eswa.2014.12.015, http://www.sciencedirect.com/science/article/pii/S0957417414007933CrossRefGoogle Scholar
  6. 6.
    Konkol, M., Konopík, M.: CRF-based Czech named entity recognizer and consolidation of Czech NER research. In: Habernal, I., Matoušek, V. (eds.) TSD 2013. LNCS (LNAI), vol. 8082, pp. 153–160. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40585-3_20CrossRefGoogle Scholar
  7. 7.
    Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270. Association for Computational Linguistics (2016).  https://doi.org/10.18653/v1/N16-1030, http://www.aclweb.org/anthology/N16-1030
  8. 8.
    Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 7–12 August 2016, vol. 1: Long Papers. The Association for Computer Linguistics (2016). http://aclweb.org/anthology/P/P16/P16-1101.pdf
  9. 9.
    McCallum, A.K.: Mallet: a machine learning for language toolkit (2002). http://mallet.cs.umass.edu
  10. 10.
    Nallapati, R., Surdeanu, M., Manning, C.: Blind domain transfer for named entity recognition using generative latent topic models. In: Proceedings of the NIPS 2010 Workshop on Transfer Learning Via Rich Generative Models, pp. 281–289 (2010)Google Scholar
  11. 11.
    dos Santos, C.N., Guimarães, V.: Boosting named entity recognition with neural character embeddings. In: Duan, X., Banchs, R.E., Zhang, M., Li, H., Kumaran, A. (eds.) Proceedings of the Fifth Named Entity Workshop, NEWS@ACL 2015, Beijing, China, 31 July 2015, pp. 25–33. Association for Computational Linguistics (2015).  https://doi.org/10.18653/v1/W15-3904
  12. 12.
    Straková, J., Straka, M., Hajič, J.: Neural networks for featureless named entity recognition in Czech. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2016. LNCS (LNAI), vol. 9924, pp. 173–181. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-45510-5_20CrossRefGoogle Scholar
  13. 13.
    Tjong Kim Sang, E.F.: Introduction to the CoNLL-2002 shared task: language-independent named entity recognition. In: Proceedings of CoNLL 2002, Taipei, Taiwan, pp. 155–158 (2002)Google Scholar
  14. 14.
    Ševčíková, M., Žabokrtský, Z., Krůza, O.: Named entities in Czech: annotating data and developing NE tagger. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 188–195. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-74628-7_26. http://dl.acm.org/citation.cfm?id=1776334.1776362CrossRefGoogle Scholar
  15. 15.
    Yang, Z., Salakhutdinov, R., Cohen, W.W.: Multi-task cross-lingual sequence tagging from scratch. CoRR abs/1603.06270 (2016). http://dblp.uni-trier.de/db/journals/corr/corr1603.html#YangSC16

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer Science and Engineering, Faculty of Applied SciencesUniversity of West BohemiaPlzeňCzech Republic

Personalised recommendations