Advertisement

A Study on Improving End-to-End Neural Coreference Resolution

  • Jia-Chen Gu
  • Zhen-Hua Ling
  • Nitin Indurkhya
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11221)

Abstract

This paper studies the methods to improve end-to-end neural coreference resolution. First, we introduce a coreference cluster modification algorithm, which can help modify the coreference cluster to rule out the dissimilar mention in the cluster and reduce errors caused by the global inconsistence of coreference clusters. Additionally, we tune the model from two aspects to get more accurate coreference resolution results. On one hand, the simple scoring function is replaced with a feed-forward neural network when computing the head word scores for later attention mechanism which can help pick out the most important word. On the other hand, the maximum width of a mention is tuned. Our experimental results show that above methods improve the performance of coreference resolution effectively.

Keywords

Coreference resolution End-to-end Neural network 

Notes

Acknowledgements

This work was funded in part by Chinese Academy of Sciences President’s International Fellowship Initiative (Grant No. 2018VTA0008).

References

  1. 1.
    Lee, H., Chang, A., Peirsman, Y., Chambers, N., Surdeanu, M., Jurafsky, D.: Deterministic coreference resolution based on entity-centric, precision-ranked rules. Comput. Linguist. 39(4), 885–916 (2013)CrossRefGoogle Scholar
  2. 2.
    Wiseman, S., Rush, A.M., Shieber, S.M.: Learning global features for coreference resolution, arXiv preprint arXiv:1604.03035 (2016)
  3. 3.
    Clark, K., Manning, C.D.: Deep reinforcement learning for mention-ranking coreference models, arXiv preprint arXiv:1609.08667 (2016)
  4. 4.
    Clark, K., Manning, C.D.: Improving coreference resolution by learning entity-level distributed representations, arXiv preprint arXiv:1606.01323 (2016)
  5. 5.
    Lee, K., He, L., Lewis, M., Zettlemoyer, L.: End-to-end neural coreference resolution, arXiv preprint arXiv:1707.07045 (2017)
  6. 6.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473 (2014)
  7. 7.
    Connolly, D., Burger, J.D., Day, D.S.: A machine learning approach to anaphoric reference. In: New Methods in Language Processing, pp. 133–144 (1997)Google Scholar
  8. 8.
    Raghunathan, K., et al.: A multi-pass sieve for coreference resolution. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 492–501. Association for Computational Linguistics (2010)Google Scholar
  9. 9.
    Durrett, G., Klein, D.: Easy victories and uphill battles in coreference resolution. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1971–1982 (2013)Google Scholar
  10. 10.
    Ng, V., Cardie, C.: Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In: Proceedings of the 19th International Conference on Computational Linguistics, vol. 1, pp. 1–7. Association for Computational Linguistics (2002)Google Scholar
  11. 11.
    Bengtson, E., Roth, D.: Understanding the value of features for coreference resolution. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 294–303. Association for Computational Linguistics (2008)Google Scholar
  12. 12.
    Haghighi, A., Klein, D.: Coreference resolution in a modular, entity-centered model. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 385–393. Association for Computational Linguistics (2010)Google Scholar
  13. 13.
    Clark, K., Manning, C.D.: Entity-centric coreference resolution with model stacking. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (vol. 1: Long Papers), pp. 1405–1415 (2015)Google Scholar
  14. 14.
    Fernandes, E.R., Dos Santos, C.N., Milidiú, R.L.: Latent structure perceptron with feature induction for unrestricted coreference resolution. In: Joint Conference on EMNLP and CoNLL-Shared Task, pp. 41–48. Association for Computational Linguistics (2012)Google Scholar
  15. 15.
    Björkelund, A., Kuhn, J.: Learning structured perceptrons for coreference resolution with latent antecedents and non-local features. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), pp. 47–57 (2014)Google Scholar
  16. 16.
    Martschat, S., Strube, M.: Latent structures for coreference resolution. Trans. Assoc. Comput. Linguist. 3(1), 405–418 (2015)Google Scholar
  17. 17.
    Wiseman, S.J., Rush, A.M., Shieber, S.M., Weston, J.: Learning anaphoricity and antecedent ranking features for coreference resolution. Association for Computational Linguistics (2015)Google Scholar
  18. 18.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  19. 19.
    Pradhan, S., Moschitti, A., Xue, N., Uryupina, O., Zhang, Y.: CoNLL-2012 shared task: modeling multilingual unrestricted coreference in ontonotes. In: Joint Conference on EMNLP and CoNLL-Shared Task, pp. 1–40. Association for Computational Linguistics (2012)Google Scholar
  20. 20.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  21. 21.
    Turian, J., Ratinov, L., Bengio, Y.: Word representations: a simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394. Association for Computational Linguistics (2010)Google Scholar
  22. 22.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  23. 23.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)Google Scholar
  25. 25.
    Lee, K., He, L., Zettlemoyer, L.: Higher-order coreference resolution with coarse-to-fine inference, arXiv preprint arXiv:1804.05392 (2018)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.National Engineering Laboratory for Speech and Language Information ProcessingUniversity of Science and Technology of ChinaHefeiChina

Personalised recommendations