Skip to main content
SpringerLink
Log in

Development and Analysis of a Sentence Semantics Representation Model

  • Published:
Cybernetics and Systems Analysis Aims and scope

Abstract

The authors overview an efficient and simple representation model of sentence semantics in the context of a paraphrase identification problem. A dependency tree is chosen as the main structure to represent connections between words in a sentence. To represent word semantics, pre-trained word representation models are used. Based on these two key components, several features helping to precisely identify paraphrases are designed. The conducted experiments proved this model to be efficient. The model application results are quite close to those of state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv:1301.3781v3 [cs.CL] 7 Sep 2013. URL: https://arxiv.org/pdf/1301.3781.pdf.

  2. R. Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, A. Torralba, R. Urtasun, and S. Fidler, “Skip-thought vectors,” in: C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems 28 (NIPS 2015), Neural Information Processing Systems Foundation (2015), pp. 3294–3302.

  3. B. Dolan, C. Quirk, and C. Brockett, “Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources,” in: COLING’04: Proc. of the 20th Intern. Conf. on Computational Linguistics (Geneva, Switzerland, August 23–27, 2004), COLING (2004), pp. 350–356. URL: https://aclanthology.org/C04-1051.

  4. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in: C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), NIPS’13: Proc. of the 26th Intern. Conf. on Neural Information Processing Systems, Vol. 2, Curran Associates, Inc. (2013), pp. 3111–3119.

    Google Scholar 

  5. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: A method for automatic evaluation of machine translation,” in: ACL’02: Proc. of the 40th Annual Meeting on Association for Computational Linguistics (Philadelphia, Pennsylvania, USA, July 7–12, 2002), Association for Computational Linguistics (2002), pp. 311–318. https://doi.org/10.3115/1073083.1073135.

  6. C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., Vol. 20, 273–297 (1995). https://doi.org/10.1007/BF00994018.

    Article  MATH  Google Scholar 

  7. Z. Kozareva and A. Montoyo, “Paraphrase identification on the basis of supervised machine learning techniques,” in: T. Salakoski, F. Ginter, S. Pyysalo, and T. Pahikkala (eds.), Advances in Natural Language Processing, FinTAL 2006; Lecture Notes in Computer Science, Vol. 4139, Springer, Berlin–Heidelberg (2006), pp. 524–533. 10.1007/11816508_52.

  8. C. Fellbaum (ed.), WordNet: An Electronic Lexical Database, MIT Press (1998). 10.7551/mitpress/7287.001.0001.

  9. R. Mihalcea, C. Corley, and C. Strapparava, “Corpus-based and knowledge-based measures of text semantic similarity,” in: AAAI’06: Proc. 21st National Conf. on Artificial Intelligence (Boston, Massachusetts, July 16–20, 2006), Vol. 1, AAAI (2006), pp. 775–780.

  10. T. K. Landauer, P. W. Foltz, and D. Laham, “An introduction to latent semantic analysis,” Discourse Processes, Vol. 25, Iss. 2–3, 259–284 (1998). https://doi.org/10.1080/01638539809545028.

  11. A. Finch, Y.-S. Hwang, and E. Sumita, “Using machine translation evaluation techniques to determine sentence-level semantic equivalence,” in: Proc. 3rd Intern. Workshop on Paraphrasing (IWP 2005) (Jeju Island, Korea, Oct 11–13, 2005), Jeju Island (2005), pp. 17–24. URL: https://aclanthology.org/I05-5003.

  12. K. Y. Su, M. W. Wu, and J. S. Chang, “A new quantitative quality measure for machine translation systems,” in: COLING’92: Proc. 14th Conf. on Computational Linguistics (Nantes, France, August 23–28, 1992), Vol. 2, Association for Computational Linguistics (1992). pp. 433–439. 10.3115/992133.992137.

  13. S. Nieβen, S. Vogel, H. Ney, and C. Tillmann, “A DP based search algorithm for statistical machine translation,” in: Proc. of the 36th Annual Meeting of the Association for Computational Linguistics and 17th Intern. Conf. on Computational Linguistics (Montreal, Quebec, Canada, August 10–14, 1998), Vol. 2, Association for Computational Linguistics (1998), pp. 960–967. 10.3115/980691.980727.

  14. G. Doddington, “Automatic evaluation of machine translation quality using n-gram co-occurrence statistics,” in: HLT’02: Proc. of the Second Intern. Conf. on Human Language Technology Research) (San Diego, California, USA, March 24–27, 2002), Morgan Kaufmann Publishers Inc., (2002), pp. 138–145.

  15. D. Milajevs, D. Kartsaklis, M. Sadrzadeh, and M. Purver, “Evaluating neural word representations in tensor-based compositional settings,” in: Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP) (Doha, Qatar, Oct 25–29, 2014), Association for Computational Linguistics (2014), pp. 708–719. 10.3115/v1/D14-1079.

  16. S. Wan, M. Dras, R. Dale, and C. Paris, “Using dependency-based features to take the “para-farce” out of paraphrase,” in: Proc. Australasian Language Technology Workshop (ALTA) (Sydney, Australia, Nov 30 – Dec 1, 2006), University of Sydney (2006), pp. 131–138.

  17. K. Zhang and D. Shasha, “Simple fast algorithms for the editing distance between trees and related problems,” SIAM J. of Computing, Vol. 18, Iss. 6, 1245–1262 (1989). https://doi.org/10.1137/0218082.

  18. J. Cheng and D. Kartsaklis, “Syntax-aware multi-sense word embeddings for deep compositional models of meaning,” in: Proc. 2015 Conf. on Empirical Methods in Natural Language Processing (Lisbon, Portugal, Sep 17–21, 2015), Association for Computational Linguistics (2015), pp. 1531–1542. 10.18653/v1/D15-1177.

  19. J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Sackinger, and R. Shah “Signature verification using a “Siamese” time delay neural network,” Intern. J. of Pattern Recognition and Artificial Intelligence, Vol. 7, No. 4, 669–688 (1993). https://doi.org/10.1142/S0218001493000339.

    Article  Google Scholar 

  20. Y. Ji and J. Eisenstein, “Discriminative improvements to distributional sentence similarity,” in: EMNLP 2013: Proc. 2013 Conf. on Empirical Methods in Natural Language Processing (Seattle, Washington, USA, Oct 18–21, 2013), Association for Computational Linguistics (2013), pp. 891–896.

  21. H. W. Kuhn, “The Hungarian method for the assignment problem,” Naval Research Logistics Quarterly, Vol. 2, Iss. 1–2, 83–97 (1955). https://doi.org/10.1002/nav.3800020109.

  22. K. Riesen, M. Neuhaus and H. Bunke, “Bipartite graph matching for computing the edit distance of graphs,” in: F. Escolano and M. Vento (eds.), Graph-Based Representations in Pattern Recognition; Lecture Notes in Computer Science, Vol. 4538, Springer, Berlin–Heidelberg (2007), pp. 1–12. https://doi.org/10.1007/978-3-540-72903-7_1.

  23. G. Sidorov, F. Castillo, E. Stamatatos, A. Gelbukh, and L. Chanona-Hernández, “Syntactic N-grams as machine learning features for natural language processing,” Expert Systems with Applications, Vol. 41, Iss. 3, 853–860 (2014). https://doi.org/10.1016/j.eswa.2013.08.015.

  24. Scikit-Learn. Machine Learning in Python. URL: https://scikit-learn.org/stable/.

  25. SpaCy. URL: https://spacy.io/.

  26. R. Weischedel, E. Hovy, M. Marcus, M. Palmer, R. Belvin, S. Pradhan, L. Ramshaw, and N. Xue, “OntoNotes: A large training corpus for enhanced processing,” in: J. Olive, C. Christianson, and J. McCary (eds.), Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation, Springer-Verlag, New York, (2011).

    Google Scholar 

  27. Z. Ul-Qayyum and W. Altaf, “Paraphrase identification using semantic heuristic features,” Res. J. Appl. Sci. Eng. Technol., Vol. 4, No. 22, 4894–4904 (2012).

    Google Scholar 

  28. W. Blacoe and M. Lapata, “A comparison of vector-based representations for semantic composition,” in: Proc.

  29. Joint Conf. on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (Jeju Island, Korea, July 12–14, 2012), Association for Computational Linguistics (2012), pp. 546–556.

  30. Paraphrase Identification (State of the art) URL: https://aclweb.org/aclwiki/Paraphrase_Identification_(State_of_the_art).

Download references

Author information

Authors and Affiliations

  1. Taras Shevchenko National University of Kyiv, Kyiv, Ukraine

    V. Vrublevskyi & O. Marchenko

Authors
  1. V. Vrublevskyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. O. Marchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Vrublevskyi.

Additional information

Translated from Kibernetyka ta Systemnyi Analiz, No. 1, January–February, 2022, pp. 21–30.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vrublevskyi, V., Marchenko, O. Development and Analysis of a Sentence Semantics Representation Model. Cybern Syst Anal 58, 16–23 (2022). https://doi.org/10.1007/s10559-022-00430-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10559-022-00430-9

Keywords

Search

Navigation