Weighted Compositional Vectors for Translating Collocations Using Monolingual Corpora

  • Marcos GarciaEmail author
  • Marcos García-Salido
  • Margarita Alonso-Ramos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11755)


This paper presents a method to automatically identify bilingual equivalents of collocations using only monolingual corpora in two languages. The method takes advantage of cross-lingual distributional semantics models mapped into a shared vector space, and of compositional methods to find appropriate translations of non-congruent collocations (e.g., pay attentionprestar atenção in English–Portuguese). This strategy is evaluated in the translation of English–Portuguese and English–Spanish collocations belonging to two syntactic patterns: adjective-noun and verb-object, and compared to other methods proposed in the literature. The results of the experiments performed show that the compositional approach, based on a weighted additive model, behaves better than the other strategies that have been evaluated, and that both the asymmetry and the compositional properties of collocations are captured by the combined vector representations. This paper also contributes with two freely available gold-standard data sets which are useful to evaluate the performance of automatic extraction of multilingual equivalents of collocations.


Multilingual collocations Distributional semantics Compositional semantics 


  1. 1.
    Artetxe, M., Labaka, G., Agirre, E.: A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 789–798 (2018)Google Scholar
  2. 2.
    Baldwin, T., Tanaka, T.: Translation by machine of complex nominals: getting it right. In: Second ACL Workshop on Multiword Expressions: Integrating Processing, pp. 24–31. ACL, Barcelona (2004)Google Scholar
  3. 3.
    Baroni, M.: Composition in distributional semantics. Lang. Linguist. Compass 7(10), 511–522 (2013) CrossRefGoogle Scholar
  4. 4.
    Baroni, M., Zamparelli, R.: Frege in space: a program of compositional distributional semantics. LiLT (Linguist. Issues Lang. Technol.) 9, 241–346 (2014)Google Scholar
  5. 5.
    Coecke, B., Sadrzadeh, M., Clark, S.: Mathematical foundations for a compositional distributional model of meaning. Linguist. Anal. 36, 345–384 (2010)Google Scholar
  6. 6.
    Cordeiro, S., Villavicencio, A., Idiart, M., Ramisch, C.: Unsupervised compositionality prediction of nominal compounds. Am. J. Comput. Linguist. 45(1), 1–57 (2019)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Delpech, E., Daille, B., Morin, E., Lemaire, C.: Extraction of domain-specific bilingual lexicon from comparable corpora: compositional translation and ranking. Proc. COLING 2012, 745–762 (2012)Google Scholar
  8. 8.
    Fung, P.: A statistical view on bilingual lexicon extraction: from parallel corpora to non-parallel corpora. In: Farwell, D., Gerber, L., Hovy, E. (eds.) AMTA 1998. LNCS (LNAI), vol. 1529, pp. 1–17. Springer, Heidelberg (1998). Scholar
  9. 9.
    Fung, P., McKeown, K.: Finding terminology translations from non-parallel corpora. In: Fifth Workshop on Very Large Corpora, pp. 192–202. ACL (1997)Google Scholar
  10. 10.
    Fung, P., Yee, L.Y.: An IR approach for translating new words from nonparallel, comparable texts. In: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pp. 414–420. ACL (1998)Google Scholar
  11. 11.
    Gamallo, P.: Comparing explicit and predictive distributional semantic models endowed with syntactic contexts. Lang. Resour. Eval. 51(3), 727–743 (2017)CrossRefGoogle Scholar
  12. 12.
    Gamallo, P.: The role of syntactic dependencies in compositional distributional semantics. Corpus Linguist. Linguist. Theory 13(2), 261–289 (2017)CrossRefGoogle Scholar
  13. 13.
    Gamallo, P., Garcia, M.: Unsupervised compositional translation of multiword expressions. In: Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019). ACL, Florence (2019)Google Scholar
  14. 14.
    Gamallo, P., Garcia, M., Pineiro, C., Martinez-Castaño, R., Pichel, J.C.: LinguaKit: a big data-based multilingual tool for linguistic analysis and information extraction. In: Proceedings of the Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp. 239–244. IEEE (2018)Google Scholar
  15. 15.
    Garcia, M., García-Salido, M., Alonso-Ramos, M.: Using bilingual word-embeddings for multilingual collocation extraction. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 21–30. ACL (2017)Google Scholar
  16. 16.
    Garcia, M., García-Salido, M., Sotelo, S., Mosqueira, E., Alonso-Ramos, M.: Pay attention when you pay the bills. A multilingual corpus with dependency-based and semantic annotation of collocations. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019). ACL, Florence (2019)Google Scholar
  17. 17.
    Grefenstette, G.: The world wide web as a resource for example-based machine translation tasks. In: Proceedings of the ASLIB Conference on Translating and the Computer, vol. 21 (1999)Google Scholar
  18. 18.
    Haruno, M., Ikehara, S., Yamazaki, T.: Learning bilingual collocations by word-level sorting. In: Proceedings of the 16th Conference on Computational Linguistics. COLING 1996, vol. 1, pp. 525–530 (1996)Google Scholar
  19. 19.
    Hausmann, F.J.: Le dictionnaire de collocations. In: Hausmann, F.J., Reichmann, O., Wiegand, H., Zgusta, L. (eds.) Wörterbücher: ein internationales Handbuch zur Lexikographie. Dictionaries. Dictionnaires, pp. 1010–1019. Mouton De Gruyter, Berlin (1989)Google Scholar
  20. 20.
    Koehn, P.: EuroParl: a parallel corpus for statistical machine translation. In: Proceedings of the 10th Machine Translation Summit, Phuket, pp. 79–86 (2005)Google Scholar
  21. 21.
    Kupiec, J.: An algorithm for finding noun phrase correspondences in bilingual corpora. In: Proceedings of the 31st Annual Meeting on Association for Computational Linguistics (ACL 1993), pp. 17–22. ACL (1993)Google Scholar
  22. 22.
    Lample, G., Conneau, A., Denoyer, L., Ranzato, M.: Unsupervised machine translation using monolingual corpora only. In: Proceedings of the Sixth International Conference on Learning Representations (ICLR 2018) (2018)Google Scholar
  23. 23.
    Lison, P., Tiedemann, J.: OpenSubtitles2016: extracting large parallel corpora from movie and TV subtitles. In: Calzolari, N., et al. (eds.) Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pp. 923–929. European Language Resources Association (ELRA) (2016)Google Scholar
  24. 24.
    Mel’čuk, I.: Collocations and lexical functions. In: Cowie, A.P. (ed.) Phraseology. Theory, Analysis and Applications, pp. 23–53. Clarendon Press, Oxford (1998)Google Scholar
  25. 25.
    Mel’čuk, I.: Phraseology in the language, in the dictionary, and in the computer. Yearb. Phraseol. 3(1), 31–56 (2012)Google Scholar
  26. 26.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Workshop Proceedings of the International Conference on Learning Representations (ICLR 2013) (2013). arXiv preprint arXiv:1301.3781
  27. 27.
    Mikolov, T., Le, Q.V., Sutskever, I.: Exploiting similarities among languages for machine translation. CoRR abs/1309.4168 (2013)Google Scholar
  28. 28.
    Mitchell, J., Lapata, M.: Vector-based models of semantic composition. In: Proceedings of ACL 2008: HLT, pp. 236–244. ACL (2008)Google Scholar
  29. 29.
    Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cognit. Sci. 34(8), 1388–1429 (2010)CrossRefGoogle Scholar
  30. 30.
    Morin, E., Daille, B.: Revising the compositional method for terminology acquisition from comparable corpora. In: Proceedings of COLING 2012, Mumbai, India, pp. 1797–1810 (2012)Google Scholar
  31. 31.
    Nesselhauf, N.: The use of collocations by advanced learners of English and some implications for teaching. Appl. Linguist. 24(2), 223–242 (2003)CrossRefGoogle Scholar
  32. 32.
    Rapp, R.: Automatic identification of word translations from unrelated English and German corpora. In: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL 1999), pp. 519–526. ACL (1999)Google Scholar
  33. 33.
    Reddy, S., Klapaftis, I., McCarthy, D., Manandhar, S.: Dynamic and static prototype vectors for semantic composition. In: Proceedings of 5th International Joint Conference on Natural Language Processing, pp. 705–713. Asian Federation of Natural Language Processing (2011)Google Scholar
  34. 34.
    Rivera, O., Mitkov, R., Pastor, G.: A flexible framework for collocation retrieval and translation from parallel and comparable corpora. In: Proceedings of the Workshop on Multi-word Units in Machine Translation and Translation Technology, pp. 18–25 (2013)Google Scholar
  35. 35.
    Seretan, V., Wehrli, E.: Collocation translation based on sentence alignment and parsing. In: Actes de la 14e conference sur le traitement automatique des langues naturelles, TALN 2007, pp. 401–410. IRIT Press (2007)Google Scholar
  36. 36.
    Smadja, F.: How to compile a bilingual collocational lexicon automatically. In: Proceedings of the AAAI Workshop on Statistically-Based NLP Techniques, pp. 57–63. AAAI Press (1992)Google Scholar
  37. 37.
    Straka, M., Straková, J.: Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe. In: Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 88–99. ACL, August 2017Google Scholar
  38. 38.
    Tanaka, T., Baldwin, T.: Noun-noun compound machine translation a feasibility study on shallow processing. In: Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment, pp. 17–24. ACL (2003)Google Scholar
  39. 39.
    Taslimipoor, S., Mitkov, R., Corpas Pastor, G., Fazly, A.: Bilingual contexts from comparable corpora to mine for translations of collocations. In: Gelbukh, A. (ed.) CICLing 2016. LNCS, vol. 9624, pp. 115–126. Springer, Cham (2018). Scholar
  40. 40.
    Upadhyay, S., Faruqui, M., Dyer, C., Roth, D.: Cross-lingual models of word embeddings: an empirical comparison. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1661–1670 (2016)Google Scholar
  41. 41.
    Vargas, N., Ramisch, C., Caseli, H.: Discovering light verb constructions and their translations from parallel corpora without word alignment. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 91–96. ACL (2017)Google Scholar
  42. 42.
    Weir, D., Weeds, J., Reffin, J., Kober, T.: Aligning packed dependency trees: a theory of composition for distributional semantics. Am. J. Comput. Linguist. 42(4), 727–761 (2016)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Wu, C.C., Chang, J.S.: Bilingual collocation extraction based on syntactic and statistical analyses. In: Proceedings of the 15th Conference on Computational Linguistics and Speech Processing, pp. 1–20. Association for Computational Linguistics and Chinese Language Processing (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Grupo LyS, Departamento de LetrasUniversidade da CoruñaCoruñaSpain
  2. 2.CITICUniversidade da CoruñaCoruñaSpain

Personalised recommendations