Advertisement

Character-Level Convolutional Neural Network for Paraphrase Detection and Other Experiments

  • Vladislav Maraev
  • Chakaveh Saedi
  • João Rodrigues
  • António Branco
  • João Silva
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 789)

Abstract

The central goal of this paper is to report on the results of an experimental study on the application of character-level embeddings and basic convolutional neural network to the shared task of sentence paraphrase detection in Russian. This approach was tested in the standard run of Task 2 of that shared task and revealed competitive results, namely 73.9% accuracy against the test set. It is compared against a word-level convolutional neural network for the same task, and varied other approaches, such as rule-based and classical machine learning.

Keywords

Paraphrase detection Word embeddings Character embeddings Convolutional neural networks Distributional semantics 

Notes

Acknowledgements

The present research was also partly supported by the CLARIN and ANI/3279/2016 grants.

References

  1. 1.
    Afzal, N., Wang, Y., Liu, H.: MayoNLP at SemEval-2016 task 1: semantic textual similarity based on lexical semantic net and deep learning semantic model. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 1258–1263 (2016)Google Scholar
  2. 2.
    Agirre, E., Banea, C., Cer, D.M., Diab, M.T., Gonzalez-Agirre, A., Mihalcea, R., Rigau, G., Wiebe, J.: SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In: Bethard, S., Cer, D.M., Carpuat, M., Jurgens, D., Nakov, P., Zesch, T. (eds.) Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, 16–17 June 2016, pp. 497–511. The Association for Computer Linguistics (2016). http://aclweb.org/anthology/S/S16/S16-1081.pdf
  3. 3.
    Bogdanova, D., dos Santos, C.N., Barbosa, L., Zadrozny, B.: Detecting semantically equivalent questions in online user forums. In: Alishahi, A., Moschitti, A. (eds.) Proceedings of the 19th Conference on Computational Natural Language Learning, CoNLL 2015, Beijing, China, 30–31 July 2015, pp. 123–131. ACL (2015). http://aclweb.org/anthology/K/K15/K15-1013.pdf
  4. 4.
    Brychcín, T., Svoboda, L.: UWB at SemEval-2016 task 1: semantic textual similarity using lexical, syntactic, and semantic information. In: Bethard, S., Cer, D.M., Carpuat, M., Jurgens, D., Nakov, P., Zesch, T. (eds.) Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, 16–17 June 2016, pp. 588–594. The Association for Computer Linguistics (2016). http://aclweb.org/anthology/S/S16/S16-1089.pdf
  5. 5.
    Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, pp. 2333–2338. ACM (2013)Google Scholar
  6. 6.
    Ji, Y., Eisenstein, J.: Discriminative improvements to distributional sentence similarity. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18–21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 891–896. ACL (2013). http://aclweb.org/anthology/D/D13/D13-1090.pdf
  7. 7.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Moschitti, A., Pang, B., Daelemans, W. (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, 25–29 October 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1746–1751. ACL (2014). http://aclweb.org/anthology/D/D14/D14-1181.pdf
  8. 8.
    Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: Schuurmans, D., Wellman, M.P. (eds.) Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12–17, 2016, Phoenix, Arizona, USA, pp. 2741–2749. AAAI Press (2016). http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12489
  9. 9.
    Kutuzov, A., Andreev, I.: Texts in, meaning out: Neural language models in semantic similarity tasks for Russian, vol. 2, pp. 133–144 (2015)Google Scholar
  10. 10.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Pronoza, E., Yagunova, E., Pronoza, A.: Construction of a Russian paraphrase corpus: unsupervised paraphrase extraction. In: Braslavski, P., Markov, I., Pardalos, P., Volkovich, Y., Ignatov, D.I., Koltsov, S., Koltsova, O. (eds.) RuSSIR 2015. CCIS, vol. 573, pp. 146–157. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-41718-9_8CrossRefGoogle Scholar
  12. 12.
    Rodrigues, J.A., Saedi, C., Maraev, V., Silva, J., Branco, A.: Ways of asking and replying in duplicate question detection. In: Ide, N., Herbelot, A., Màrquez, L. (eds.) Proceedings of the 6th Joint Conference on Lexical and Computational Semantics, *SEM @ACM 2017, Vancouver, Canada, 3–4 August 2017, pp. 262–270. Association for Computational Linguistics (2017).  https://doi.org/10.18653/v1/S17-1030
  13. 13.
    Rychalska, B., Pakulska, K., Chodorowska, K., Walczak, W., Andruszkiewicz, P.: Samsung Poland NLP team at semeval-2016 task 1: necessity for diversity; combining recursive autoencoders, wordnet and ensemble methods to measure semantic similarity. In: Bethard, S., Cer, D.M., Carpuat, M., Jurgens, D., Nakov, P., Zesch, T. (eds.) Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016, San Diego, CA, USA, 16–17 June 2016, pp. 602–608. The Association for Computer Linguistics (2016). http://aclweb.org/anthology/S/S16/S16-1091.pdf
  14. 14.
    Saedi, C., Rodrigues, J., Silva, J., Branco, A., Maraev, V.: Learning profiles in duplicate question detection. In: Proceedings of IEEE IRI 2017 conference (2017, in press)Google Scholar
  15. 15.
    dos Santos, C.N., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: Hajic, J., Tsujii, J. (eds.) COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, 23–29 August 2014, Dublin, Ireland, pp. 69–78. ACL (2014). http://aclweb.org/anthology/C/C14/C14-1008.pdf
  16. 16.
    Segalovich, I.: A fast morphological algorithm with unknown word guessing induced by a dictionary for a web search engine. In: Arabnia, H.R., Kozerenko, E.B. (eds.) Proceedings of the International Conference on Machine Learning; Models, Technologies and Applications. MLMTA 2003, 23–26 June 2003, Las Vegas, Nevada, USA, pp. 273–280. CSREA Press (2003)Google Scholar
  17. 17.
    Wu, Y., Zhang, Q., Huang, X.: Efficient near-duplicate detection for Q&A forum. In: Fifth International Joint Conference on Natural Language Processing, IJCNLP 2011, Chiang Mai, Thailand, 8–13 November 2011, pp. 1001–1009. The Association for Computer Linguistics (2011). http://aclweb.org/anthology/I/I11/I11-1112.pdf

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Vladislav Maraev
    • 1
  • Chakaveh Saedi
    • 1
  • João Rodrigues
    • 1
  • António Branco
    • 1
  • João Silva
    • 1
  1. 1.Department of Informatics, Faculty of SciencesUniversity of LisbonLisbonPortugal

Personalised recommendations